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ABSTRACT 


The optimization of systems with several cost functionals, 
(both single and multi-control cases) have been treated in this thesis. 
While the discussion has been mostly restricted to deterministic systems, 
some results for systems with sudden changes in parameters are included. 

It is shown that the optimization of a system with respect to 
an objective function (which is expressed as a function of several 
given cost functionals) is embedded in the linear combination problem 
(i.e.) an optimization problem in which the performance criterion is a 
linear combination of the given cost functionals. According to the 
technique proposed in this thesis, the optimal controls are first deter- 
mined for the linear combination problem as a function of the weighting 
factors. A search technique is then used to determine the optimum 
values of the weighting factors for the given objective function. One 
of the advantages of this technique is that the solution of the linear 
combination problem can be used for different objective functions 
formed out of the same set of cost functionals. niece are included 


to show the application of this technique. 
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CHAPTER (1) 


INTRODUCTION 


1.1 Background 
For the past two or three decades much attention has been 

devoted to the problem of optimizing the behavior of systems. This 

has resulted in a variety of mathematical formulations and computa- 

tional techniques. Some examples of such problems are maximizing 

the range of a rocket, minimizing the fuel consumed to achieve some 

required terminal state, maximizing the profit of a business, and min- 

imizing the error in the estimation of the position of a certain object. 

Searching for a control which satisfies all the requirements and at 

the same time minimizes (or maximizes) a performance criterion con- 

stitutes the fundamental problem of optimization theory. The salient 

feature of this approach is that the performance criterion described 

by a single functional and a single control function is used. However, 

there are many real-world situations which cannot be meaningfully 

described by the above approach. A description involving a single 

cost functional with several controls or several cost functionals with 

a single control or several cost functionals with several controls 

Hol /]* 


may be necessary to describe the situation. has classified 


some of these situations. 
In recent years several investigators have attempted to 
tackle problems of this type. For an optimization problem with a single 


AL avirge i lial amas bas pti le 3 
control and several costs some of the references are f 


* Numbers in [ ] refer to references listed at the end of the Chapter. 
Note that references appear at the end of each Chapter. 
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Optimization problems of the differential game type (zero-sum games) 
in which a single cost functional and two control functions ap- 


pear are discussed intts228,9,13] 


among others. However, not much 
work appears to have been done on the optimization of systems 
governed by several controls and several cost functionals (n-players- 
games or non-zero sum games), especially from an engineering point of 
view. The references which the author has come across in this 
subject a 
1.2 Scope of the Thesis 
In this thesis it is proposed to discuss the following types 
of problems. 
a) Optimization of a deterministic system with a single 
control function and several cost functionals (see 
figure (1.1)). This problem is discussed in Chapter (2). 


b) Optimization of a deterministic system with two control 


functions (zero-sum differential game) and several cost 


functionals is discussed in Chapter (3), (see figure (1.2)). 


c) In Chapter (4), the optimization of a linear system subject 


to a linear quadratic performance criterion is discussed. 
Two antagonistic control functions will be used. It will 
be assumed that the parameters as well as the weighting 


matrices in the performance index are subject to sudden 


changes in values. This problem will be tackled as stochastic 


version of a special case of the problem discussed in 


Chapters(3). 


d) The optimization of a deterministic system with several 
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System Model: X 


Gost Scales: 


Search for U*eU such that 
Ce een eaten ee) (ney 
VUcU 


Usis. the set of admissible controls 


Objective 


Figure (1.1) Optimization of a System with a Single 
Control Function with Respect to an 
Objective Function (minimization problem) 
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Given 


System Model: x 


Gost Scales: 


Search for U*eU and V*eV, such that 


b(J,9+++5Jy) 5 O(Jys+++55y) < o(Jys06. 


(U,V*) (U* , V*) 
VYUeU and VeV 
UGG save~n Sets) Of padmissible controls 


Objective 


(U*,V) 


Figure (1.2) Optimization of a System with Two Control 
Functions, with Respect to an Objective 


Function (Min-Max Problem) 
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control functions and several cost functionals (non- 
cooperative games) is discussed in Chapter (5), (see 
fisure .(Pss)): 
The format of each chapter is essentially the same and 
consists of the following: 
a) Summary of previous work and scope of the rest of the 
chapter, 
b) Mathematical formulation. Here necessary conditions 
for optimality are obtained using Pontryagin's 
Maximum Principle, 
c) Discussion of computational techniques used, 
d) Application of the technique to a few examples, and, 
e) Discussion of results. 
Only the titles and the broad outline of the problems 
considered in this thesis are mentioned here. A more detailed 
description of each of the problems and its relationship to 


previous work in the area will be found at the beginning of each 


chapter. 
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Figure (1.3) Optimization of a System with p-Control 
Functions and p-Objective Functions. 
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CHAPTER (2) 
OPTIMIZATION OF A DETERMINISTIC SYSTEM WITH A 


SINGLE CONTROL FUNCTION AND SEVERAL COST FUNCTIONALS 


2.1 Background 

One of the fundamental requirements for any optimization 
technique to be effective, is that the optimality criterion must 
some how reflect in an analytic form and in proper proportions all 
of the factors which are apparently important to a given system. 
Several approaches have been suggested for the problem of optimiza- 
tion of a dynamical system with a single control function and 
several cost functionals. Some of them are summarized below. 


[7] 


Nelson has proposed the following technique (see figure 


a Let us desig- 


2.1). Suppose we have several cost scales oe N° 


nate the most important cost by J Then the problem is to search 


N° 
for the control function U*(t) which minimizes Jy subject to the 


constraints (J, Su Bus Ket O15, bueilie B's are constants) at final time. 


The key to this approach lies in the choice of J,, and the constants 


N 
B'S. which may not always be an easy task. However, on the whole, 
this technique is considered a useful approach. 
(ae has proposed what he calls "Hierarchical Optimiza- 


tion Criteria" and his technique can be explained as follows, (see 


figure 2.2). The given costs are ranked in order of importance as 


Jy> Joreeeodye We first optimize with respect to the primary cost Jy 


and get the optimum value of this cost say Vi: The second step is to 


by V, + AV, where AV, > 0, and optimize the system with 
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Given 


System Model: X = £50) 


Costa leaves: ) 


Search for U*eU, such that 
J, (U*) < J(u) 


VUeU 


Objective 


Figure (2.1) Nelson's Technique 
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System Model: xX 


Given 
Search for U*eU, such that 
Objective - n? 
v4 J, @ ) < J, @) 
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| 
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System Model X = £(X,U) 
J, (WU) — Vy rip AV, 
Given 
Jy Seen D AV 4 
Search for U*el, such that 
i A a GES as me 9) 
Objective N nf vied N 


VUeU 


Figure (2.2) Waltz's Technique 
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12 
respect to the second cost Jy» at the same time ensuring that 
J, @) Saar na AV,» and get the optimum value of the second cost say Vo: 
The third step is to change Vy by an increment AV» and optimize with 
respect to the third cost with the restriction that J, WU) Syxe) Ge AV, 


and J, W) SS + AV, and so on. This technique is essentially a 
constructive approach to choose the B.'s in the previous technique 


but the disadvantage of the present method is the increments AV AV. 


[etarves 2 N? 


must be chosen by the designer and this is a matter of engineering 
judgement, moreover we have N sequential optimization problems. 


[12] 


A novel approach is due to Zadeh in which he has intro- 


duced the term 'Vector-Valued Cost". Chang has used Zadeh's idea 
and obtained a generalized version of Pontryagin's Maximum Pineda 
See figure (2.3). In this approach Chang ends up with a problem in- 
volving the minimization of a linear combination of the given costs 
with all the weighting factors being positive. Unlike the previous 
methods, all the cost scales are considered simultaneously instead 
of one at a time. This is an improvement. However, to the best 
knowledge of this author, Chang has not described any technique to 
determine the weighting factors C'S: These factors must be known in 
order to implement the optimal control strategy in any practical 
application. 

The work reported in the remaining portion of this chapter 
although closely related to Chang's work goes further. The optimal 
control is determined for a performance criterion which in turn is 


expressed as a function of several cost scales. This criterion will 


be referred to as the "Objective Function". The basis for choosing 
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the objective function is discussed later on. The author believes 
that this is a more useful approach compared to the previous 
techniques. 

In practice it may be necessary to determine the optimal 
control for a given system with respect to several objective func- 
tions, based on the same set of cost scales. It will be shown that 
there is no need to solve the optimal control problem for each 
objective function separately from the beginning. 

Given the various cost scales J, 's the optimal control for 


k 


the system is first determined with respect to an objective function 
N 


amare 


fs Cys where the c's are treated as parameters, (the so called 
k=1 

linear combination problem). The c,'s may be positive or negative. 
The optimal control is expressed as a function of the c's. The 
specific values of the cs are then determined by a search technique 
for each objective function. This will be explained further later on. 


Once the linear combination problem has been solved, the 


results can be used for any objective function expressed as a function 


of the given cost scales. 


2.2 Mathematical Formulation of the Optimization Problem 


Description of the Problem 


Let the dynamics of the system to be controlled be repres- 
ented by a set of n first order ordinary differential equations, 


Xt) = E(X(t)y UCL) 7; oats 


M0) =% | 
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continuous in X and U, and continuously differentiable with respect 
to X. X is an n-dimensional vector representing the state of the 
system! U,is the control vector of the system, U has a dimension r- 
The contro’vector is required at each instant of time to satisfy 
certain boundedness constraints of the form, 

0 Auge +s Ww) ie Geet Sacre (232) 
where a, 's are positive constants. 
Pieeconceo! LUNCE1ON UCL) as,called an admissible control if 
i) it is piecewise continuous on the control interval [t. Tl], 

where bY is the initial time and T is the final time, and 

d4) it satisfies (2.2), vte[t ,T]. 
The set of admissible controls is denoted by UYU. Moreover it is assumed 
that the state variables are required, to satisfy the following 

Pe) =20 
at t = T. wW(Q) is a real non-negative scalar function of its arguments. 
It is further assumed that the N cost functionals are expressible in 


the form, 
oi 


Sa g,, (X(T) ] +/ L, [x (t) U(r) Jdt Se 


ie 
e Os 


|e ae a ea 
where 9,8 and Ls are all real non-negative continuous scalar 
functions of their argument, the Z 's are all continuously different- 
iable with respect to X, and the 91.8 are of class car The B's are 
positive constants. 
[6] 


Following the formulation of Knapp and Forst , the coer 


functionals in (2.4) can be put in the following form, 
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9g, [X(t) ] 


1 5 


z(t) = 2, [X(t) uct) ] + | ae dest X(t) UCE ae 

Z(t.) = 9, [X(t,)] | (255) 

Rae yw aie gM | 
where prime denotes "transpose". It is clear that 

J, (4) = Z,,(T) ; ee AL aN | C26) 


The z(t)'s are positive non-decreasing functions of time representing 
the "state of the cost scales" in the system. The reason for intro- 
ducing this transformation will be clear as the discussion proceeds. 
Let us define the following function 
o = (21500652) 
¢ is of class nar We shall refer to ¢ as the "Objective Funetton". 
In practice we associate a penalty function Pe with each cost scale Zp 
This p.f. is expressed as a function of the deviation of the actual 
value of the cost scale and some specified desired or nominal value. 


In other words 


P, Py (21, - Zg)s oe CT ee a 
where all the P's are of class ea These P'S may have the dimen- 


[4] 


sion of dollars or some other criterion such as desirability The 


objective function ¢ is obtained by adding all P,'s 


b= > wy 

Moreover, let us define the following target set B as 

B PLAX ZO Kees, WC LEMO., Z, (T) ar Be, Ait Toe. NER (2°57) 
The Control Problem can now be conctsely stated as follows, 

Find an adnissible control function UX(t)eU, such that 


KP x? U*(t))eB 
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1] 
and 
o[T; Xx Ux(t)] < o[T; x Ult) |, 
VU(t)eU 
It is worth noting that, a time-dependent system can be easily trans- 
formed to a time-independent system, by introducing an extra state 


variable as, 


X41 (0) =l1, x 


tt ‘5? is S 


Therefore it is sufficient to consider the time-independent systems. 
The reason for introducing the transformation (2.5) is that we can 
cast the problem in a form for which the necessary conditions can be 
; . [9,10] 
obtained very easily using the formulation of Rozonoer ; 
Necessary Conditions for Optimality 
We shall hereafter write X, U, Z instead of X(t), U(Ct), Z(t) 


respectively to simplify the notation. The control problem can be 


described by the following system of n+tN+1 "augmented state" equations, 


X = £ CX U):, n-equations 
‘ 991 (X) 
z= L,. (XU) + eee £(X,U) = g,.(%,U), N-equation 
een EN rorge | 
$ = ab g(X,U) + eee pes 2 CX, U)), l-equation 7 (2.8) 
OZ4 1 OZ N 
and 
X(t) = Xo 


= = Pdr rae gh 
Zz (to) g, [X(t JI; k We 5] 
o(to) = [2 (t5) +++ 2 (t,)] 
assuming the final time, T, is fixed. Written in this form, this 


problem becomes a special case of Rozonoer's problem in which the 


objective function is a linear combination of the augmented states. 
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18 
In the present case the objective function is one of the states, 
namely 9. 

We shall follow Rozonoer's procedure to obtain the necessary 
conditions for optimality using Pontryagin's Maximum Principle. We 
shall consider the free right end point problem first. Let us define 
the Hamiltonian as, 


He =iP' £(%.Uy Pat g, (X,U) ‘ee osteo (XU) 
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where P is an n-dimensional vector. We get, 
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where the c's are constants (assuming Ch F710), Uae hore ah) 
Since 

Peat) = 0, Keak oetecs N (2.15) 
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Before we proceed with the rest of the main derivation, we 


will digress briefly to comment on the cS: The tangent hyperplane 


N 
to the surface 6 = constant is oe CL Zr at t=T. 
k=1 
The expression for the Hamiltonian becomes, 
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the expression for H becomes, 
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Pee c lean trom (2218) andj. 10) that, 


N (2.20) 
~ 99, (X) 
aS ii me et = t=T 
k=1 


The preceding results can be summarized in the form of a theorem. 
Theorem (1) A necessary condition for optimality of U*(t)eU in 
system (2.8) ts that the Hamiltontan defined in (2.19) attatns an 


absolute maxtmum for every t, t_ <1t< JT. This optimal control 


O 
wtll be a funetton of the C'S, aefined by (2.16). 
For the following system 

XC) = ACE)XG) + OVE) I (222m) 
where A(t) is nxn matrix and 2 is a vector-valued function of 
dimensionality n, the following holds 
Theorem (2) A necessary and suffictent condttton for optimality of 
U*(t)eU in system (2.21) ts that the corresponding Hamiltonian attains 
an absolute maximum for every t, t <t< 7%. 
Turning now to the problem of constrained end point; assume that the 
final state X(T) satisfies the condition 

p[X(T)] = 0 


Then the boundary conditions in (2.20), have to be modified to 


N 
by ay) (2.22) 


P(T) Ste 3G 
k=1 
where a is a positive constant. 
It T is not fixed this will require that, 
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at t=T. Moreover if some of the state variables are fixed at t=T 
the corresponding adjoint variables are not specified at t=T, this 
may be applicable to the cost scales Z's too. 
2.3 A Search Technique for Determining the Optimum Control 

We have already pointed out, for the case where there are 


no constraints on the cost scales, that as a consequence of the 


condition for optimality, 


Cre Loe eG (2.16) 
k tet 


We find that for a given objective function 4, its optimal value at 


t=T occurs at the same point in the Z-space as the optimum value of 
N 


ye CL 2) at t=T. Since it is easier to handle the optimization 


k=1 
problem in which the objective function is a linear combination of 


the cost scales, the optimal control for the problem under discussion 
can be determined in the following manner. 
Step (1) Consider the optimization problem with the same system equa- 


tions and same boundary conditions as for the given problem 
N 


— 
but with objective function > CL Zh where the c,'S are 


k=1 
parameters. The details involved in determining the optimal 
control U* as function of the c's are best explained by means 
of a numerical example. Therefore further discussion of this 
step will be postponed to a later section. At this stage, 
for the purpose of explaining step (2), we will assume that 
U* has been determined as a function of the c's. 


Step (2) This step is concerned with the search of the optimum values 


of the c's which minimize the given >. If there are direct 
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22 
relationships between the various cost scales then this search 
can be carried out in the Z-space. Otherwise the search will 
have to be carried out in the C-space. There may arise situa- 
tions where a modification of these techniques will have to be 
used. These points will be clarified further by the numerical 
examples which may be found in a later section. At this stage 
it will be possible only to describe the basic concepts in- 
volved. A flow chart describing this procedure is shown in 
figure (2.4). In what follows we are going to consider the 
case where c,'s are all positive. The extension to cases where 


k 


we have +ve and -ve c's is straightforward. Let us denote 
the set of points in the Z-space corresponding to all possible 
variations in c,/cy; | =peankegN ss atlt=t byea. 
-Z-Space Search 

In this search direct relations must exist between the 
various cost scales in terms of the ratios c,/cy» j= Ziad, Ns) OChe r= 
wise we use the C-space search. For the Z-space search, let us pick 
one of the cost scales as a dependent variables, say Z4> and the re- 


maining N-l scales as independent variable. The expression of the 


gradient of ¢ is expressed as, 


oe (39/8z,) (92,/92,) T (36/3z,) 
[grad $] = = (Qe24) 
| 94/82, (36/824) (924/92) + (36/32,) 
At any point Zen, the tangent hyperplane to 7 is described by 
N 
” c,2, = constant (225) 
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Figure (2.4) Proposed Optimization Technique 
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Differentiating (2.25) successively with respect to 24S: Dele eta ths 
we get 

924/02, = -c,/ey, Ta ose ahi : (2,26) 
Ssupstituting (2.26) into (2.24), 

(36/82, )(-c,/c,) + (36/22,)| 
[grad 9] = ; C22 
| (96/925) (-ce,/c,) + (96/32, 

All the components of the gradient vector are known at any point in 7. 
It must be noted that if the objective function 4 attains its minimum 
at Z*eT, then the components of the gradient vector are zeros. 
Therefore, setting equation (2.27) = 0, we get 

(99/02,)/ (26/82, ) = Conde ad dhe lees (2-28) 
It is clear that these are the same as the conditions given in (2.16). 
The problem now reduces to the determination of the minimum of a 
function assuming that its value and its gradient at each point of 
the search region are known. In this regard many techniques grouped 
under the broad category of steepest descent methods are available. 
The methods used in the examples of this chapter are the "Bi-Section 
Search'' for one dimensional search and the "Accelerating-Step Search" 
[8], for the 2-dimensional search. 


C-Space Search 


If there are no direct relationships between cost scales in 
terms of c,/cy; J = 2,...,N, the previous method cannot be used. Then 
we use C-space search. In this search the components of the gradient 


of » are given by, 


- ae | ak Re 
as a PL aa, 
; 7 . - - b 
sve = E a's ‘OF dosqes"s tle vsvinsrssue” ; 
i mais iis 


(ef.9) Vn vs ig * i (eee 


aS Sh ‘tint ok 
Ay sbyne) + ioe seh 

Vacs) | - 
cee ag Ga \y Se) ty 0] 


of Sh te hog Vos 36 were Big hy hace 7BY | Pare es ae ae 
mgtinin eat aakarie 9° sultsogl sy bjoat eo say a, si beter 


1et3S8 ats | ratjhev! srs bbe dh t¢ biti h: og 


bia iY ; 
198 ou Ly =i ORS wi haps at = ~ 
_ oS 0 a a i i a “4 oes 
(Ag Np Wy owe i? oN Nae cseteon ee A 


(84.8) ob wows enoksibies pie ea oes eitt fain need 2 


a 


& to ‘mrmimtet Sd Yo hoiawnkucrgaah one 98 on 
ae bes aa 
ty a ps 


20 TAitog Hons Ie iae thang re Bon} suk oe 
Naquot, faupintoss yet PARSE, ict 
Ii deli ava ots bouton ju0 960 ieee 


my he oak nee 4 atts ‘eign pha The as ‘i 


nt also deon asausad \atiehdizetaa sion (ba! ae 8 
’ 
nant .bsai oad Joanho bodstain mart a igeces sHb 


25 
[grad 9] = 


peeere’ Ciaeiieiea)? ere (96/92) (82, /3(c,/c,)) | 


| ° . 
| (86 /92,) (02/0 ep/e,)) erent 5 Air (9¢/92z.) (dz ,/9(e./c,)) | 


(2529) 
All the components of the gradient vector are known if c,/ey 
j = 2,...,N, are specified. This is the case for each POL te Ze ins. 
The gradient technique can be used to determine optimum values of 
(c,/e))*, j = 2,...,N, which minimize the given $4. 
If the number of the cost scales becomes very large, the 
gradient technique becomes unwieldy. We can use what is known as the 


[2]. 


"Random Search" technique The use of the techniques involving 
Z-space search and C-space search will be illustrated by the examples 
that follow. 
2.4 Examples 

The purpose of this section is to illustrate the application 
of the theory and computational technique presented earlier in this 
chapter. Three examples will be discussed. The first example will 
deal with a system with two cost scales. The second will consider the 
same system with three cost scales. In these two examples the use of 
the Z-space search technique will be demonstrated. The concluding 
example is intended to illustrate the use of the C-space search technique. 

The single-axis satellite attitude control problem is one of 
the most popular examples used to illustrate optimal control techniques. 


Since it is a good illustration for multi-cost control problems, it 


will be discussed in all examples in this chapter. The equation of 
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motion of the satellite about its centre of gravity, under the in- 


fluence of gas-jet controllers with limited thrust, is given by 


2 
eet he iy Ged IL(t)| <M 30) 
at? } 


where 9(t) is the attitude error angle, I is the moment of inertia, 


and L(t) is the control torque limited by the maximum magnitude M. 


Defining the normalized variables, 


yas ) 

x,(t) = = e(t), 
its 
x, (t) = y Ct)» and @ 3h) 
ucey s= 2 Ct)/M 
the equations of motion and control constraints are written in the form, 

e \ 
x(t) = x,(t), x(t.) = X49 | 
X(t) = u(t), XR) =ix55 ' (2.32) 
| u(t) | ed and te[t),T] 


whene *E\is the «final. time: 
Example (1) 
Consider a system with dynamics given by equation (2.32) and 


the following two cost functionals, 


T 


J, = ip dt eo and 
" | 
* (2233) 
J, = f | u(t) | dt 
t 
oO 


where J is the time elapsed and J, is the fuel consumed. The object- 


ive is to reduce the attitude and its derivative to zero while minimiz-— 
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27 
where ¢ is some specified objective function. 


Following the discussion in section 2.2, the augmented state equations 


are, 
x, (t) = x,(t), x, (t,) = X19 
x(t) = u(t), x, (t_) =x 
Z 
2240 20 (2.35) 
z,(t) = 1, Zz, (t,) = 0 
Zo (t) =~ (uct) |, Z, (t.) = 0 
and 
Fae aiest tele f 
T is free, and x, (T) = x, (T) = 0 
Since the given system is time invariant, we can set te = 0. 
In this example two objective functions will be used, 
‘ J rf 
(i) (2 >Z) = 2, 2 where r > 1 (2.36) 
tie x Z 
Ci) $ (2 Zo) = az, +z. where a > 0 (2375) 


Step (1) Solution to the Linear Combination Problem 
This step is common to both the objective functions. Con- 


Sider the objective function C424 + CoZos The Hamiltonian is given by 


H = PyX> + Doe aes || (2.38) 
And 

; OH } : : : 

Paar 3x, = 0, p, (1) undefined since x, (7) is fixed (2799) 

: oH : : : 3 

Po Ox, See Py (T) undefined since x, (T) 1s) fixed 42.40) 


We shall first begin by assuming that Xoq = 0, because it leads to a 
symmetric control law. Moreover the Hamiltonian is identically zero 
with respect to time, since we are dealing with a time invariant system. 
Et as obvious ‘that T = z,(T). Let us designate z, (T) by F. Then after 


some manipulations we arrive at the expression 
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28 
u*(t) = DEZ[(1tc,/c,) (2t/T-1)], VEEL st Cran) 


where DEZ means "Dead Zone Function" Figure (2.5) l4], We also get 


2(1+c,/c,) 
Ze 
DA pam mersmeses ol OY) X19: Xoq = 0 (2.42) 
Vi1+2(c./e.) 
Zeek 
2 = 
F omapeaa vi X40 Xo = 0 (27543) 
142(c,/c,) 
hese, worthnotine that if (c,/c,) = 0, we get 
T= Fes 2VX14 ; Xoq = 8) (2.44) 
which is the case of minimum time. If Cc, ? O, we get 
T > @ 
; x = 0 (2.245) 
F>0 20 
The equations of the switching curves in the X 1X5 Space are given by 
coe (Opp (co icy) el x5) as 
and (2.46) 
x, = 0.5 x, x, | 


These switching curves are shown in figure (2.6). These switching 

curves are obviously valid for initial states (Xj: Xo) with Xo # 0. 
The expressions for the time elapsed to reach the state 

(0,0) from any initial state Cae oy Oke and Cope got ee and the 

corresponding fuel consumed as functions of co/e, and the initial state 

are given by 


ah 2 | ) 
Tee 2(1+c,/c,) 0) eee yee | 


(2.47) 
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Figure (2.5) Optimal Control for Example (1) - (Chapter (2)) 
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Figure (2.6) Switching Curves for Example (1) (Chapter (2)) 
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For these cases equation (2.47) becomes 


Tait ACs Vo gall a the, Te) 
fee (ie Sed OM AO Oya Er ean 
F=1+2 \V3/(2+4e,/c,) me - 
5 
er (2,48) 

T = 2(1tc,/c,) \V 2/(2+4c,/c,) 

eh Ont ‘(x Feb, ox =0) 

bene P Ge 20 

F=2 /2/(2+4c,/c,) } 


The relation between F and T, as cy/c, varies, for the given 


intial) conditions’ is given in’figure (2.7). These relations are 


identical with the ones obtained by Neteon’ 4 for the given initial 


conditions using another method. The realization of the control law 
eiven aa (2.41) is shown win figure (2.8). 
Step (2) 

We shall now proceed to determine the optimum value of 


cy/cy; which will minimize the objective function 


o=2 ae . where tao4l (2.49) 
pe? 
The two cost scales z4(T) and z,(T) are directly related because, 


from equation (2.47), we get, 
LS ee 
Tia ¥ + (on fey) \ { (2x4 9#899)/ (0. Stey/ey) 


or (2750) 
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We can use the Z-space search technique described in section (2.3). 


Zo is taken as dependent variable and Zz, as the independent variable. 


The 'Bisection Search" technique will be used to solve this one 


dimensional search problem. 
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From this point, the search technique will be carried out 
for X16 = Xo = 1 and several values of r. The initial values for 


Z and Z, are taken to correspond to Cy/cy = 0.  Frompequation (2.48), 


setting c,/c, = 0, we get, 


Zee 2 = 1 V6. when ore, = 0 (2.51) 


By assuming a reasonable range of values of z the minimum of 


ie 
(2,525) is located. Table (2.1) shows the optimum values of co/c, 
for different values or r. The sensitivity of 6 with respect to 
variations in (c,/c,) for different values of r is shown in figure 
(279). ~The. variation ot (c,/c,)* with respect to r is shown in 
figure (2.10); 

Before proceeding to consider the second objective function, 
it is interesting to compare the results obtained for the first 
[5] 


objective function with the results of Kalinin who also used the 


same objective function. It can be seen from table (2.2), that our 

results are identical to those of Kalinin's method for initial states 
on the x,~axis. However, for initial states not on the x,~axis, our 
values of $ are smaller than those obtained by using Kalinin's method 


(see table (2.3)). It would therefore appear that the generalization 


made by Kalinin to include initial states not on the x,~axis is not 


valid. 


Minimization of the Second Objective Function 


We shall now consider the objective function ¢$ = azy +2: As 
pointed out earlier, it is not necessary to repeat step (f)2. Even “in 


step (2), only some algebraic computations are needed. The results 
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Figure (2.10) Variation of (c,/c,)* with respect to r for $=2525 
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38 
for this objective function are shown in Table (2.4), and the sen- 
sitivity analysis is shown in figure (2.11). The variation of (c,/c,)*; 


With respect to a is shown in figure (2.12). 
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Table (2.4) 


Example (2) 
Consider a system with dynamics given by equation (2.32) 


and the following three cost functionals, 


Jy = it dt | 
: | 
FL: 


{ utr) fda, and > (2.525) 
0 
eo 2 

J, = J TREY Gum ale 


0 
is the time elapsed, Jy is the fuel consumed, and J, is the 


Cy 
i} 


where Jy 
energy dissipated. The objective is to reduce the attitude and its 


derivative to zero while minimizing 


O(5yI5 Ig) fy 


where ¢ is some specified objective function. 
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In this example two objective functions will be considered, 
Pepe tar) 
(i) $ (2) 25524) = | +25 tZ4 9 and 


C1) o (2) 525923) = 10 (24-2) "+z +2, 


ye 
where 
Za) =] z,(0) = 0 
25(t) = |u(t)| 2(0) = 0 (253) 
z(t) = es z,(0) = 0 


Step (1) Solution to the Linear Combination Problem 


This step is common to both objective functions. Consider 
the objective function > = C424 -f- CoZ5 Se C4235 for this objective 


function the Hamiltonian takes the form 


2 
H = pjX, + Pyu - cy - c,|u| - yu (2.54) 
where 
° oH ; : : : 
Si ere a 0) p.(T) undefined since x,(T) is fixed 
Lf Ox) at it 
22 55)) 
° oH : : ; : 
aes ax, = “Pp, (t) p, (7) undefined since x, (T) is fixed 
From equation (2.55), we get 
p,(t) = w | 
t (2.56) 


‘ 


Py (t) = —w,t + Wo | 


where Wy and W, are unknown constants. The expression ford in (2.54) 


becomes 
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The control u*(t) which maximizes H is given by 


u*(t) = SAD[-w)t + Wo 3 Nyon] (2.58) 
where 

ny = cle, s and 

Sat 2(c,/c,) + (c,/c,) 
SAD means ''Saturation-Dead-Zone" function, see figure (2.13). For 


a derivation of (2.58) see the Appendix at the end of this chapter. 
We are going to consider first the case where X19 > O and Xog = OF 


Moreover from equation (2.57) the Hamiltonian is identically zero 


Vee 0,7). | Then) from Ho = 0, we get 

Wy = -(1 + cy/c, oe c4/c,) (23,59) 
poovicing wnat W(0) 1s —-1. Similarly for Heer = 0, we have 

w= -(2/T)(1 + cy/cy + C4/c1) (2.60) 


providing that u(T) = +1. The expression of the control law given 
by equation (2.58) becomes, after substituting equations (2.59) and 
(2.60), 
u*(t) = SAD{[( + c,/c, + c,/e,)(2t/T - 1)];. Ny ony} (2.61) 
23.61) 
From equation (2.61) we can derive expressions for Ty>To> and Ta, see 


Fieve (28) 3)., 


atet.= Ty> 

-w4T} - (1 + co/cy + c4/c,) = -(ce,/e, + 2c,/c,) 
We get 

t,/T = 0.5(1 - c,/ce,)/(1 + cy/cy + c,/c,) (2.62) 
And at t = Ty 1 To 
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-0 (ty + T,) - (1 + cy l/c, + c,/c,) = “c,/Cy 
then, 
t,/T S (c,/c,)/(1 + Cy/cy + c,/cy) (2.63) 


imi = aad [ 
Similarly at t T 9 + 3? 


~w, (1, +1,+ 4) - (1+ o/c, + c/c,) = Co/cy> 
then 


efi (C2) C/G +res/c.0t o/c.) (2.64) 
8 Dra: Ee eAD Siatails 


Figure (2.14) shows a typical trajectory passing by (x5 929) reaching 
the origin. By solving along such trajectory we obtain the following 


relation between the final time and X10? 


Ite, /ctc.,/c 
it poe ee ee (2.65) 


sce eee 10 
\f2Cte,/e,te,/c,)-3{c4/c,) -1 


By substituting equation (2.65) into equations (2.62), (2.63), and 


(2.64) we get 


0.5(1-c./c,) 
(Are sue fend SE a ee 2VX19 (2.66) 
\/ 2ite, /eyte4/e,)-$c4/e) 1 
ca/cy 
a ee ee 2VX4 0 C571) 
\ [2 (1te,/e,+e4/¢1)-H(c3/¢,)°-1 
Cy/cy 
= 2Vx (2.68) 


e a set eg 4 A ee ee 10 
\/ 2(1te,/e,te4/c4-3(C3/C7) -1 


It is necessary that 0 < n/c, =< 1, so that Ty is always +ve. 


Using equations (2.66), (2.67), and (2.68) we can obtain expressions 


for z, (T) = T, Zz, (T) = F, and z(T) = Ei 
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Figure (2:14) A Typical Trajectory for Example (2) (Chapter (2)) 
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(HS so sate erste < dae Gaseanenmnt aromas 27X10 (2.69) 
HE 2 
Gee feel fe,) -1 
if 
eee 2YX1 9 (2.7/0) 
- if 2 
\J 2(1te,/ce,te4/c,)-Fle,/c,) -1 
1-(c,/c,)/3 
z,(T) eee ee ee 2K C2277 19) 
if uv 
\/ 2(ite,/c,te,/c,)-F(c,/c,) -1 
Rewriting expressions Ty> Tos and T, given ani(2 00) u(2 0s sand 
(2.68) as 
T) = OYX 9 
To BYxXI5 (2.72) 
enna: 10 j 
The equations for the switching curves are given by, 
=) ~~ 
ae, (l/a” - 0.5)x,|x,|> 
x, = [(1 - 8/2 - B°/3 - GB) /G + B/2)"]x5 |x, 15 (2.73) 
= a a) em i oe ere tee ae 
x, = [0 - a7/2 - B°/3 - ab - By/2 - ay)/(a + 8/2) ]x, |x,|, 
and 
x1 = 0.5x,|x,| 


see figure (2. 
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Figure (2.15) Switching Curves for Example (2) (Chapter (2)) 
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iS (c4/c,)/3 ee? 
Fe eee ae RR ~119 x 


E = *Xoq + 
\/2cate /c,+c.,/c ene /c Se Aah 
ales cer be i) Sor at a 


where +tve sign is taken for (X19 2X oQ)EC and -ve sign is taken for 
+ 

JeG , see figure (2.15). 

Rewriting equation (2.73) as, 


x, = ax, |x| 
ee | 
: : Q (2375) 


Mary x A x 


D ale and | 
= 0.5x, |x, | J 


The realization of the control given in (2.561). is shown in figure 


(6) stits as, valid only for thesanitial states 90) &F and 


AO 
+ 
(X49 2X a9) EG 5 
As a typical initial condition, let Xgl, and XoQ70- For 
this initial state, equation (2.74) becomes, 
= 2(1tc,/c,t+e,/c,) 


1 2 
2(1te,/e,te,/c,)-3(c4/cq) -1 


F = : Se (2.76) 
\/2dte,/e,+e3/e,)-Fle4/eq) “1 


2(1-(c,/¢,)/3) 


E = Sees 


2(1te,/e,te4/¢,)-4(C4/¢,) -1 


For the given initial state (1,0), the relation between z,(T), z,(T), 
and z.(T) as Cy/Cy> and eno vary, is shown in figure (2.17). 


It is interesting to note if cy/cy = Ca/Cy = 0 we get the 
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(1.225,0.816, 2.449) 


z5(T) 


Figure (2.17) Relationship Between z4(7), Z(T) and z,(T) in 
Example (2) (Chapter (2)) 


oil 

minimum-time problem and if cy/cy = 0, we have the case of linear 
combination of time and energy, and if Ca/ Cy = 0, we have the case 
linear combination of time and fuel which is the case considered in 
example (1) of this section. 
Step (2) 

We shall now proceed to determine the optimum values of 
co/cy and c4/c,, which will minimize 

d (21525523) = 21 te, +24" 21a) 
The three costs Z1» Zo> and Zz are directly related because from 


equation (2.74) we get, 


e,/c, = 3[1 - (247% 59) / (ZX 99) 1» aad (2.78) 


= 3(z ) + (z WiC ars (2.79) 


37* 99) / (29-X 99 axe 


We can use the Z-space search technique described in section (2.3). 


is taken as dependent variable and z, and z, are taken to be 


- 2 3 


1 


independent variables. For specific values of the coordinates Z5 
and Z4 we can get the corresponding Co/cy and c4/c, by using equation 
(2.78) and (2.76b). The “Accelerating Step Search" technique will 


10 2? XoQ70- The search consists of two stages. 


In the lst stage, we define increments AZ. and Az. and evaluate the 


be carried out for x 


= ae sae = Lijiesesil 5 where Mil 
value of 9» at (pxAZp qxAz.), p Hi q ; is 


P 
and M Az, are bounds on Zo and Z. respectively. We locate the minimum 
q 
of » with respect to these discrete values ising i sure (2..1/)).o bane 


ing from this point the "Accelerating Step" search technique is used 


to reach the minimum of ¢ within reasonable accuracy. The results 


obtained for this specific > are 
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z,*(T) = 2.31601516 
z,*(T) = 1.18737496 
2% (T) = 1.01407607 
@2(T) i= 7.80213579 
* = 

(cy/c,) 0.51268013 
(c,/c,)* = 0.43785380 

0.00000000 
[grad 9] = 

0.00000001 


The sensitivity of ¢ with respect to variation in (c,/c,) 
and (c,/c,) is shown in figure (2.18). 
Let us consider the following objective function 

feer IOs 2200) 7b a (2.80) 

1 a 3 

As mentioned in the previous example, there is no need to formulate 
a new problem. Just evaluate > at the previous discrete values and 
locate the minimum. This consists merely of algebraic computation. 
Then using the "Accelerating Step" search technique using the gradient 
of the new ¢ to get the optimum values of cy/c, and Ca/c). The 


results of this specific ™% are: 


Za =) 2516897420 


i 
Zo = 1.36258676 
Z3% = 1.22818886 


ok = 2.87629843 


Cosy ie = 0.29590315 
fee) * = 0.29590316 

0.00000002 
[grad 9] = 


0.00000001 
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The sensitivity of ¢ with respect to variations in (c,/c,) and (c,/c,) 
is shown in figure C2110 ie 
Example (3) 

The purposes of this example is to illustrate the application 
of the search technique in the C-Space (the space of weighting factors). 
The same dynamics and cost scales as given in example (2) will be used. 
Although the cost scales in example (2) are related (in terms of cy/c, 
and c,/cy) equations (2.78), and (2.79), these relationships will not 
be needed when the C-Space technique is employed. 

The computations carried out in step (1) of example (2) are 
applicable to this example also. Step (2) is of course different since 
the search for the minimum value of ¢ will now be carried out in the 
C-Space. 

Following the discussion in section (2.3), for the objective 
function 


2 2 2 
) (2552592) ie ou +25 +23 (2.81) 


we get, using equation (2.29), 


[grad $] = 


224 (92,/8(c5/c1) +225 (825/28 (Co/Cq)) +224 (824/38 (co/cy)) pas 


224 (924/38 (C4/C1)) +225 (825/38 (c4/¢1))+224(82,/8 (c4/c)) 


10 ~ 0% Xog = O. “From (2.76) and figure (2.17) sabe 


components of the gradient vector are known. Note that the relation- 


Assuming x 


ships between the cost scales were not used. Using the "Accelerating 
Step-Search", the results obtained are, 


z* = 2.31601518 


( che? hos (yo\.a) nb erotaningy "3  aorate: eae. 
) ee 


he iy. y 

ae ¥ rat 
miigohiogs ofa atesseull bp ob BE alyosipa aha ty cour one 4 
/CaadoR? narsiaisw to eases. SFT saR ge) Sri} wk xt siamo shek dS rs 


A 


69 ad Driv (S) ofqumtsoee avg an da lese aan uo eae 


pPAgS) FO) wiegs2 HEY baystos ste (8) Gerais mt elena haan ace a 
tou Ebiw eyidanoisele> sesnd.00N yf) hie Ur FY) ‘sletoeaie 

. -bs¥ilqme a2 aap lnil: a OES? az, oe | sing 

are (5) oars’ to CL). qeta ak 906 beiwms anit) basque wal 
avgle 1persttrh servos To 2h (8) qo Orts SiPpaeen Sidt He 
218 ok gud felriss od you Tike + Yasaka stinker ban of aol 


iS 


Midas te ‘Sy We! : (P5) eateoae mt hia, St, av seal 
/ ; %. aie yeu 4 ve 


C18.) . . a ae wa ty paar 
ty vee sage 
= (¢ ae 


i al 8) oe) oeN Nuch) ly! 
isis: || [SENT ARIA cola ea 
. Byte | 


bi ss0hetay ati? piles 
) ttgegfosan” ads neat 


aD 


} 

10(z,-2)° fez tomy 

xis WO 
10 X97 0.0 
8 
6 
4 
2 

LON: 3) eza) 


, eee 6, 4.082) 
el 


= 


z(T) 


Z 
Figure (2.19) Variation of ¢=10(z,-2) t25+24| a with respect 


to z,(T) and z.(T) 


2 we 
a 1 


=< 


Woshnaat wijis 


iy 
¢ 
i] i 
v 
4 iy t rE 
, J 
/ D 
Bi ae S 
, 
v 
rat 
he: 
: | ir 
, 09 
1 
9 
7 
t 
/ 
j ne 
j r é 
f. : : , 
/ mi 
ee | = 
aT i 
‘ 
/ / 
if 
fa ‘ae 
j f / 
ee — a = wa / PA - 
te ee Seine > neers 
a 
} if WV p™ ; 
= wr wf i i be | 
ra y v 
, a any * ( 
| CG i ati J ei 0! ull 
a y 7 } 7 wt 7 ; 4 
Ya ; — a 
J = ie 
- a ; 0 - wt 
{ ee 7 wre 
le i A 
gat a rt 
a. a f 
ae a Pa 
ae Lot @ a 
yw pal A 4 
wee x ' ra ' 
_ an ee” f a 
a - 4 
ra ’ 


oj 

Cd 

———— 
ae, 


> 


SAE oo ai 
Fee (S- whiOttey Te Fobsolbivay 
= on year : 


2! = 


he 


iar _ 
(29. Sobor. CP ee. 
BY, < bmn, Cg 


— 


56 


N 
+ 
i] 


1.18737492 


N 
% 
! 


= 1.01407608 


7.80213578 


-o- 
+ 
Il 


0.51268031 


(c,/c,)* 


(c,/c,)* 0.43795372 


0.00000023| 


[grad ¢] 
0.00000002 


which agree within six decimal places, with the results obtained 
previously. 
For. 


2 
¢ (24 2Z5 92) = 10(z,-2.0) + Zo oA zZ. (2783) 


and using the same initial conditions, the expression, 


[grad ¢] = 
F 20(2,=2) (82. /9(c,/c,)) + 3z,/3(c,/c,) + 32z,/3(c,/c.) 
aay Sg Soa 2 Ds): 3 Oak Ma 
20(2,-2) (82,/9 (c,/c,)) + 92z5/9(c4/cy) + 324/3(ca/cy) 


yields the results 


z4* = 2. 16897416 
Z5* = 1.36258702 
z4* = 1,.22818877 
o* = 2.87629843 
* = QQ. jal 
(c,/c,) 0.295902 
(c,/c,)* = 0.29590386 
-0.00000121 
[grad 6] = 
~0.00000014 


which agree within five decimal places,with the results obtained before. 
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APPENDIX A-1 


Derivation of the SAD Function 
ee ee ee AY FUNCTION 


Given: 
£(t).U(t) - a,.|u(t)| - a,.u?¢e (Al.1) 
where U(t) is a piecewise continuous function on the interval 
Ont |: |U(t) | < 1, and f(t) is defined on the interval [0,T]. 
ay and a, are +ve constants. 

Required: 


Find U(t) which maximizes expression (Al.1). 

Derivation: - 
Let us assume U(t) is +ve, then expression (Al.1) becomes, 
f(t) U(t) - a, .U(t) - a,.U°(t) (A1.2) 
Differentiating (Al.2) with respect to U(t), we get 


£(t) - a, - 2aU (t) (ADS) 


A 
We set (Al.3) equal to zero and solve for U*(t) 


Ute 


a A e 
(£(t)-a,)/2a, | (A1.4) 
Bue O;s-U=(t) =< 1.) Therefore, 


habe hah FS ener U*(t) is zero CATS) 


ns 
andedt citb) a,t2a,, U*(t) is +1 
Stnularly for UCt) <= 0; we get 
for £(t) > —a, U* (Ct) 1s. zero 


andor. (t) < -(a,t2a,), U*€p): as7=2 (Al.6) 


Define the following quantities 


aa 
= (AL.7) 
No ay + 2a, 
Then U*(t) which maximizes (Al.1) is given by 
(A1.8) 


Ux(t) = SAD [f(t); nyongl 
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r 
1 £(t)>n, 
£(t)-n, 
= Ws sf CO) <n 
£(t)+n, 
= Tie De) =n 
No-Ny 2 rate iB 
=i £(t)<-n, J 


This is* shown in. figure (Ad.1). 
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Figure (A-1.1) Saturation and Dead-Zone (SAD) Function 
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CHAPTER (3) 


OPTIMIZATION OF A DETERMINISTIC SYSTEM WITH TWO 


CONTROL FUNCTIONS AND SEVERAL COST FUNCTIONALS 


oe. introduction 

In this chapter, the objective is to determine optimal controls 
for a deterministic system with two independent controls, with respect 
to a performance criterion which in turn can be expressed as a function 
of several cost scales. The controls will be assumed to be antagonistic 
in the sense that if one control minimizes the performance criterion 
tne-;other control, would try to maximize it. 

Several examples where such a problem would arise can be 
given. For instance, in the field of economics, the problem of maximiz-— 
ing the productivity of an economy can be formulated along the above 
lines. The action taken to maximize productivity is treated as one 
control whereas another factor which adversely affects productivity is 
taken to be an antagonistic control. In the area of guided missiles, 
the Pursuit-Evasion problem is another example. 

To the best knowledge of the author the previous work in this 
area appears to be mostly confined to cases where only a single cost 
functional of the min-max type is used as the performance criterion. 
This problem has often been formulated as a Zero-Sum Two-Person Differ- 


ential Camel l»2»3+425] 


A limitation of the available techniques should be mentioned. 
If it is required to determine the optimal controls of a system for 
several objective functions of the min-max type, then the optimal 


control problem must be solved separately for each objective function. 
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In some cases, the determination of optimal controls by the available 
techniques is very difficult [example ¢2) 14],. In other cases, the 
available techniques do not work at all. This point will be illustr- 
ated later in the examples. 

However, the above difficulties can be overcome by following 
the procedure described in the previous chapter. It will be shown 
that there is no need to solve the optimal control problem for each 
objective function separately. As described in the previous chapter, 
the two step procedure is used except now, we have to search for a 
saddle point instead of a min or max since we have a min-max type object- 
ive function. 

3.2 Mathematical Formulation of the Optimization Problem 
Dynamics of the System 

The dynamics of the process to be controlled are represented 
by a set of n-first order ordinary differential equations, 

KECK UY) (an) 
where (x = dX(t)/dt). X is an n-dimensional vector representing the 
state of the system. U and V are the control vectors of the system 
and are of dimension Ty and ty respectively. f is an n-dimensional 
vector valued function, defined over a suitable bounded, connected, 
open set D of (X,U,V) space, and is assumed to be of class c? On Mats 


domain of definition. The following information about the system is 


available to both the controls, 


i) the initial value Xx, of the 
state 
ii) the dynamics of the system, 


iii) the system state, Xf) Be times. 
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The control variables are constrained as follows, 


U(t) © RCE, 
u (3.2) 


V(t) ¢ KE. 
V 


where Ky» K, are convex and compact, E represents Euclidean space. 
These control variables U(t) and V(t) are chosen according to a set 


of rules as follows, 


U(X) 
mele) 


U(t) 


V(X) 


V(t) 


We 


where U(X) and V(X) are members of classes of functions denoted by 
U and V respectively, which are piecewise c? and the control variables 
satisfy (3.2) Vte[t.>T], where t is the initial time, and 1 is) the 
final time of the process. The significance of equation (3.3) is that 
we are interested in obtaining closed loop optimal controls. 

A terminal manifold S, which is an n-dimensional, connected 
manifold of class c? in X-space is assumed to be given. The process 
is to be terminated whenever XeS. Points on S will be denoted by Xn 


Cost Scales and Objective Function 


Let us define the following cost scales, 
T 


J, (X,U,V) = Gy, Xp) +f 1, (KU, V) dt (Sou) 


Ct 
oO 


2 P 
where the functionals 91's are assumed to be of class C and defined 
on S and L's are of class c? and defined on D. Let us define the 

; Z 
"Objective Function" as (Sy o+++sIy)s where ¢ is of class C’ and 


defined on S. U is trying to maximize $ while V is trying to minimize 
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A pair of controls (U,V) eUxV is defined to be a "Conststent 
Pair", if corresponding to these controls equation (3.1) has a continuous 
solution defined over some interval [t\ Tl], satisfying the initial 
condition at ce and Xe. We will consider only those sub-classes ucU 
and vjcV such that for every vel, and veV, the pair (U,V) is consistent. 
Statement of the Optimal Control Problem 


The optimal control problem can now be stated as follows: 


Find a Consistent pair (U*,V*) such that 


X(T3X 19 U*, Ve) ES (3.5) 
and 

6(X, UV) < o(X,U*,V¥) < 6(X,U*,V), (3.6) 
VX, Ue, and VeV, 


a 


(U*,V*) is defined to be an Optimal Patr of control variables. 
X*(t) resulting from (U*,V*) is called Optimal Trajectory. 


Necessary Conditions for Optimality 


Following the discussion of section (2.2) we get, 


’ 991, 1 5 
z,(t) = Gx f + Le j | 

= ooh 
Z,.(t.) 91, (%,) a ) 
Kat Les we gh 


Let us adjoin these variables to the state variables to 


obtain 


x<e 


’ 

: f X(t.) | Xo 

rs , , ri (3.8) 
QS e+ L} j2(e.)  [60x,) 


Ne 


where Z,G and L are N-dimensional vectors. The objective function 


¢(Z) can be rewritten as 
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The second term in the right hand side has no effect on the optimiza- 
tion problem since it is a fixed quantity. 
We have now cast the problem under discussion into a format 


[2] 


for which the procedure used by Chattopadhyay can be used. The 


Hamiltonian is given by 
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P He (7) 
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X = Hp X(t.) = Xo 
and 
Hy = Hy = 0, between corners. 


Here we allow for the fact that the trajectory X*(t) may have corners. 


2 
We assume that X*(t) is of class C between corners. 
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These constants may be positive or negative. Then the expression of 


H becomes, 


N 
at mae ea 
= + ——— 
ie eee ee ct eral, 
k=1 k=1 
Let us define 
Lai aude hee ied 
Bee ILE 
k=1 
Since P(T) = 0, we have 
. y 09), 
p(T) =) GR| 
k=1 


From equation (3.14) and (3.11) we have 


Where the expression for H becomes 


N 
~ fas | 
<2 ' 
H= Pf + - ce 
k=1 


(3.513) 


(3.14) 


(Calais) 


C3 sei) 


(3, 7) 


Thus we see once again that the problem under discussion with object- 


ive function $(z) is embedded in the problem with objective function 


N 
\ c,z, where the c,'s are parameters to be determined. 
co kk k 
k=1 
the previous necessary conditions we get, 
N 
& BY aaa 19), 
P(t) = -Hy, pa) =) Ge aa 
k=1 
X(t) = He» X(t.) = xX, 
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Hy = He = 0, between corners (3719) 


Equation (3.19) covers the four possibilities min min 4H, min max H, 
U. yy teeny 
max min #, and max max H. In the present discussion we will be 
Ue oc WER 
interested in max min H. If there are magnitude constraints on U and 
Ug. S$ 
V then the necessary conditions given by equation (3.18) and (3.19) 


become, see [5] 


N 
pss im a & 091, 
= - H ~U% aye zs LES 
P(t) = -(, + Hove ++ Heva), = PCT) i oo Mi 
a (3.20) 
ae ae 
where N 
H(X,P,U,V) = P'f + i eh. 
k=1 
and 
H(X,P,U,V*) < H(X,P,U*,V*) < H(X,P,U%,V) (3.21) 
Vx. Ue, and VveV, 
3.3 Example 
Let us consider a system with dynamics given by, 
x= ax + butbiv (3922) 
u Vv 


where 
Stas = x and 2,..b;°b). >’ 0. The final time 7 is fixed.) here are 
fe) ol egekttca) 7 
no constraints on the control variables u and v. It is required to 
find the optimal controls u*(x) and v*(x) such that 
1 1 
r ir ; 5 3 fi 3 “5 
Min Max x? os | V a - J us dt (S235) 
vs ux T L : t 
The conventional method cannot be applied, since the objective function 
Let us reformulate the problem 


is not in the form of a single integral. 


as follows. Given the system, 
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and the cost scales 
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Determine the optimal controls u*, v* such that 
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Following the discussion in section (3.2), we get 


x = ax + bu + bv x(t) my 

a = x(ax + bu + bv) z(t.) = x°/2 
25 = 0-5 uw Z(t.) = 0 

fs ee AG v z(t.) = 0 


The objective function is C424 + CoZ5 ot C424 
The Hamiltonian is given by 


H(x,p,u,v) Beat ices bu + b.v) 


2 2 
+ 0.5c,u a5 0.5c,V 
From equation (3.18) we get, 
p(t) = -ap(t) p(T) = ¢,x(T) 


From equation (3.19) setting 


H =H = 0 we get 
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where Cy 280: 5 Co 40, C4 Fi 
Co is -ve to insure the maximization of H with respect to u, C. is 
to insure the H# is minimum with respect to v. It is a well known 
technique to assume 
p(t) = K(t)x(t) (Ce 
Therefore equation (3.29) becomes 
ut = —C/e,) Bene) x(t) : 
ve = -(1/e,) bUK(t)x(t) 
where the scalar K(t) is the solution of the Riccati equation 
R(t = CDeeCL) “Aen be Felder db A 1kK? (2) (3. 
oid 8 SU KY, 
with the boundary condition 
Rien cr | (3. 
Then es Ais 
Wey TK Ce) GRE) t= 2a) Pak Ct) = ‘) dt (3. 
K(t) 3 
Rereeal= Iie oun Heya 7 
where A = Co)d C3)b, 
The solution of equation (3.34) becomes 
1 
Ze Se of 
K(t) 2acy B + (2a-B) Exp[2a(t-T) ] ( 
where 
2 2 
B= (c,/cy)b,, SP (c,/c,)b, 
or 
R 
= ———— 3 
K(t) “1 1. 2). Exp(2a(t-T)) 
where 


R = 2a/B 


Then equation (3.31) becomes, 
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By substituting this equation into (3.22) and solving for x(t), we get 


a 7 
1 + (R-1) Exp[2a(t-T) 


a oo ula weno 1 + (R-1) Exp[2a(t -1) 1 | ee 
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The realization of the control variables is shown in figure (3.1). The 
relation between Z1s Zo» and Z. at t=T for different values of (c,/c,) 
and (c,/c,) is shown in figure (3.2). There are some bounds on (c,/c,) 


and (c,/c These can be obtained by setting u(t) = constant for any 


D: 


value of t, we obtain a relation between (c,/c,) and (c,/c,). 
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see figure (3.3). Similarly from the expression of v(t), we get 
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Figure (3.2) Relationship between z4(T), z5(T), and z,(T) in the 
Example Chapter (3) 
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(c9/c}) 


Figure (3.3) Bounds on (c,/c,) and (c,/c4) 
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For — = 0 we get line (1). 


For & = © we get line (3). 


Having solved the linear combination problem and determined 
us and vk as functions of the ratios c,/c, and Ca/cy we now proceed 
to search for the optimum values of these ratios which correspond to 
the objective function given by (3.23). This search can be performed 
in the Z-Space or C-Space depending on whether the cost scales and the 
ratios cj/cy; j = 2,...,N bear any direct relationship to one another 
or not. In the example under consideration, we shall use Zz, as the 
dependent variable, Zo and Z. as the independent variables. Since 

2 
- - Pu oly (3.42) 
3 De (c,/c,) 

we can carry out the search in the Z-space. Im the previous chapter 
we considered problems of pure minimization or maximization of the 
objective function. In this chapter, however, we are considering min- 
max type problems. Consequently we have to look for a saddle point of 
the objective function rather than its minimum or maximum. Consequently 
the search technique described in Chapter (2) will be modified to take 


Enis-into account. 
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All the components of the gradient vector are known at every point 

in 1, figure (3.2). The procedure for locating the saddle point 

consists of the following, 

1. We specify increments Az, and Az. and compute ¢ at the set of 
points AZaXP > Az,xq, poe oly eM 4 gq = 1,0... Mieewhere tl Ag and 


P q p 2 

ae are bounds on Zo and Z. respectively. We locate the saddle 
point of ¢ with respect to these discrete values. 

2. Starting from this approximate saddle point, we use a modified 
form of the "Accelerating Step Method" to get as close as possible 
to the exact saddle point. The criterion used to determine winning 
points is the norm of the gradient should be minimum. For the 
starting point A the winning points are B, C and D. If the start- 
ing point is A', the winning points are B', C' and D'. The tech- 
nique is explained in figure (3.4). 


The results obtained for the given objective function, 


od = 224 + V2Z. - v2z. (3544) 


and T = 2.0 are 


(c,/c,)* = 0.0 


x 
(c,/c.) SeO83705L2 


0.00000000 

| | 
[grad 9$]= | | 
| -0.00000000 | 
and the value of the objective function = 16.58586876. All the 


numerical results reported in this chapter are up to 8 significant 


figures. 
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Now let us consider the same system with objective functions 


l 
N 
at 


aca me © ERY 
2 
Z. wake 88 0 


le age ah zy + 4.0 


(3.45) 


There is no need to formulate the new problem from the 


bee nning m+Step, (1) and part of step (2) (up to the stage of discret- 


ization) are common to all objective functions. The approximate value 


of the saddle point for each of the two new objective functions is 


determined with respect to the discrete values mentioned earlier. The 


gradients are, 


06, "] 4 } 
iL 2 
32, | | ancey/e,) = Z./(Z,+0.5) 
[grad 9, ]= (=| 
| | 
3b5 wag 
325 | “i (ey/e3) oo 1/(z,+0.5) 
(3.46) 
d>, | a 
— | / 1/(e/e5) - (2.¢2) /(25¢4)* | 
be as eee 
[grad oo] iy | 
d¢ 
2 | | -1/(c,/c) + 1/(z,+4) 
OZ. | [ | 


The search is then carried out using these gradient vectors. The 


results obtained are for 


Z 


3 

at z+ 0.5 

k= 7. 84308 
(c,/c,) 72. 358 

k= 7". 060 
(c,/¢3) 7.57405 : 

-0.00000001 

[grad oe 


+0.00000003 


| 
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Value of the objective function 


Z + 2 
a ee TT 
2. 
* = 
(c,/c,) 6.79863103 


(c,/c,)* = 9.56304898 


0.00000000 


[grad ¢,] = 
0.00000000 


Value of the objective function 


= 9 205 359620 “ang sfor 


= 8.94178680. 
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[3] 


[4] 
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CHAPTER (4) 
OPTIMIZATION OF A TWO-CONTROL MULTI-COST 


LINEAR SYSTEM WITH JUMP PARAMETERS 


4.1 Introduction 

In this chapter, the objective is to determine optimal con- 
trols for a linear system with two independent controls and several 
cost functionals and subject to sudden changes in its parameters. The 
controls will be assumed to be antagonistic in the sense that if one 
control minimizes the objective function the other control would try 
to maximize it. The objective function is taken to be the Conditional 
expectation of a linear combination of several cost functionals with 
weighting factors unity, positive or negative. 

Such problems would arise in situations where there is a 
possibility of failure of some components of a system or sudden shifts 
in environment. This problem can be considered to be a stochastic 
version of a special case of the problem discussed in the previous 


chapter. 


The related work for a single control can be found in [4]. 


4,2 Mathematical Formulation of the Optimization Problem 
Dynamics of the System 


The system to be controlled is described by the linear vector 


differential equation 


ECE) eub(E atic) we Beast va Baliey grab) 
(431) 


X(t.) = Xo 


where (X(t) = dk(e\idt)- x(t) ds an n-dimensional vector representing 


the state of the system. U(t) is an ry -dimensional vector and V(t) 
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is an r,-dimensional vector are control variables. It is assumed that 
the number of possible parameter values is finite, say s. We shall 
represent this variation in parameters by a state Markov jump process. 
We will assume that each element of the random matrix BAGG), By ft) » 
B.(t) ] is a separable Markov process and that there exists ans xs 


matrix Q such that the conditional probability, 


Pr tlACE+A) By (tts) By (ta) ]=[A,»By, By} | (A(t) .B,(t) BL (t) I=[A, »B if 


vi? yi 


da Ch) AS OCA) ne | 
anal beste 


1 + q, 4,604 ROCA) fh eee = er aS 
We denote 


Patra Coy, 8 Co)’, B Co) 1=[A, 5B, 6B )IRP, 2 f= Ts. 23s (4.3) 


Where P is an s-dimensional vector. 

For obtaining the optimal controls, the quantity of the data 
fed back to the controllers must be specified. It will be assumed 
that the feedback signal will contain both time and the instantaneous 
state, i.e. (t, X(t)). Furthermore, since the parameters of the plant 
vary in a random manner, an attempt might be made to monitor these 
variations. Suppose that there are m sensors on the plant with out-~ 
puts given by u(t). u(t) is an m-dimensional random process. This 
latter process need not be Markovian nor need it be continuous, but 
it will be assumed to be bounded. It will also be required that u(t) 
is independent of the control variables. Therefore the following in- 
formation about the process is available to both controllers, 

i) the initial value xX, of the state 


vector 


ii) the state X(t) of the system at time t, 
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iii) the dynamics of the system 
iv) the vector u(t) 
It will be assumed that at any timet, the controllers select U(t) 
and V(t) based on their observation of the vector CECE ctr te 


In other words 


W(t) = ulesx(t) uw) | 
(4.4) 


1 


V(t) = V(t,X(e) u(t) | 


The hat above U and V will be used to emphasize the difference between 
the function and its value. In other words U and V represent the 
controllers and they are therefore non random, while U(t) and V(t) are 
the output of the controllers and are thus random processes. The 
controls U and v are bounded and continuous every where, for t <t< T. 
The class of admissible control pair is defined as those U: U(t) = 
U(t,X(t) ,u(t)) and Vi V(t) = V(t,X(t),u(t)). These classes of admiss- 
ible control pairs are selected to ensure that the system differential 
equation will be meaningful and integrable when U and V are inserted 
in equation (4.1). 

Cost Scales 


Let us define the following quantities, 
1e 


Ah [ L {1 X(t ,t 2X, 2U,V) UCT X(t 5t, XK 9UsV) su (t)) 


[e 
O 


V(r X(t 5t, 9X, U,V) u(t) dt (4.5) 
where 1, (t,X,U,V),k = 1,...,N, are non-negative and continuously 


differentiable with respect to X,U and V. 


Objective Function 


Let us define the objective function 
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N 


7k ale = 7, e 
OCU, V5t »X su(t.)) EL) | Spruit, x uted) tok eu(t)} (4.6) 
k=1 


whe re ils +1tor -1, kis ys e.3N, depending oniithe problem underhand. 
Tis the final time and is fixed. E{ } is the expected value of the 
quantity between braces. U is trying to maximize $ and v 1S sCryine sco 
Minimize 9. 
Statement of the Control Problem 

The optimal control problem can now be stated as follows, 
Find the optimal pair (Us, V*) such that 

bCU,VESE XK sult,)) < o(UK, VSL, Xu (tI) 

< (U,V 5t,.X u(t) (4.7) 

¥ Ue and VeV 
The trajectory K*(t5t, 4X, 5U*, VA) resulting from (Ux, V¥) is called 
"Optimal Trajectory". 


Necessary Conditions for Optimality 


Bixei(t xe velo, Dink Ez Define a vector valued random 
oO 
variable P(t xX.) as the solution to the following ordinary differ- 
Oo 


ential equation, 


d nw aw 
dt P(t,X(t3t, »X,U*,V*)) 
S a 4 1 
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If we define the Hamiltonian to be of the form 


yn jaceox + BY(t) U(t,X,u(t)) 
HCC x, U,V) Ph). . 
+ Bo(t) V(t,X,u(t)) 
N 


zt » cy.2y (taXsU(t, Xu (t)) ,V(t,X,u(t))) (4.9) 
k=1 


then equation (4.8) can be put in the same form as equation (3.20). 
It must be emphasized that in the stochastic problem under discussion, 
the initial value of the Hamiltonian must be specified. 
aco FB Ce DUCA sn.) 
= t 
A(t »X >U,V) =P (t 5 »X,)- . 
L + Cyne Ege) 
N 
- _ e 
es 


k=1 
where UcU and VeV. 


fy KU Ce (a), VCC i ee eo 


Based on the results obtained in chapter (3) for the deterministic 
case and the results obtained by Sworder [4] for a similar problem 
with a single control, we make the following statement as a necessary 


condition for optimality. 


"We can find optimal controls U*eU and V*eV such that 
B{H(t,X,U,V*) [t,X,u(t)} < ECH(t,X, U*, V8) |t,X,u(t)} 
< E{H(t,X,U*,V) |[t,X,u(t)} (A541) 


VUeU and VeV." 


“a 
na 


This means that U* maximizes, and V* minimizes, the conditional expect- 


ation of the Hamiltonian. The above statement can be formalized as a 


theorem. But this will not be attempted here. We proceed to discuss 


the applications. 
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4.3 Application 


To illustrate the application of the necessary conditions 
obtained in the previous section. Let us assume the system in (4.1) 


has three cost scales. 


\E 
z4 -+ | eae x(a) yas 
ie 
Oo 
ie 
el ' 
= aie (Ul (TR, Ult)) dt (4.12) 
Be 
Ut 
Zo a Oe ee Vir) )dt 
iG 
oO 
Ere eek 
a a 


where 
R is an n x n positive semidefinite matrix with constant elements, 


Ry is an ry x Ty positive definite symmetric matrix with constant 


elements, 

Ry is an rx Ty positive definite symmetric matrix with constant 
elements. 

Let the objective function be 


(2, 22523) = Et (2, +24725) pop tok ou(t,) 3 (4.13) 
[4] 


From here on, we follow Sworder's work for a single control closely 
and obtain similar results for the case of two controls. The same 
assumptions made in [4] are made here also. However for sake of being 
self contained the assumptions will be restated here at the appropriate 
places. 


As a consequence of equation (4.11), see [1], we have 


yt Sea CEE ee 


if + : roe = ap | i j 21 ‘4 *. " . 7 
+e J ‘tnt heey ~< ( ; - 7 as a TF idle 7 


PeyaWwol Calm Sinn sagen z ba fi aie i 


eas has.eransr Su) LT iw six} natal as 
eg 


iy pa 
7 i : 
ath 
¢ ) 3) 
i] —— 
' } Orie : y Set at i 
t fe ' aye T a0 | i mil -t 
' * 
i 
t 
{ ’ = 
* ‘ 
~~ 4 
. 
| 
} 4 
ivy f Li/ } 
he S . 


f 7 = 
ra | 4 
’ i 
j 4 i a = 
ww 
2 T 2 
i — ae 
gi 


oo itn *Dpae sya obi avi heod on: AK n 7 2 
An, i] 
‘rH CN "Eras <a aan uy b y a 
ISG) DERISMO PS Seo l 2s ov MP ERY Th ne ¢ 
y i ; i by is al 
ao 


i) t ime . : 

A Pa y j a ao 7 

A eee 44 riokTone? evbfaetde i 
oe i 5 : » 


abe bh bh od. ata ess rea vobing moe a0 4 1 om ext 
i a) a ; iw as i a) ; 


PAGS owl de 8669! 57 oo” 2iteney ee iietdo bas 


a oe 


— 


“i 


nl ae? i 
sic lo ai 


87 


U*(t,X,u(t) ) 


B{R, BY (t) P(t,x)|t,X,u(t)) 


Fe (4.14) 
V¥(t,X,u(t)) 


il 
i 


-E{RY BY (t) P(t,x) [t,xX,u(t)} 


It is necessary to find an explicit relation for P(t,X) in terms of 

the vector (t,X,u(t)). It must be noted that the information available 
to the controllers is sufficient to determine the instantaneous values 
Of ACE), By (t)» and BU(t). Since the sample functions of the elements 
of A(t), By (t)» and BY (t) are not continuous, they are independent of 
the control law. Therefore, the equations for the optimal controls 
become, 

i 


Re By (t) EGP (esx i Eee) 


Uk(t,X,u(t)) le 
oe | 
Ro 


I 


VE (t,X,u(t)) Bit) E{P(t,X)|t,X,u(t)} 


From equation (4.8) we get, 


S— P(t, X(13t,X,U*,VA)) 


A(t) 
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= RN (4.16) 


where 
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To determine the unique solution to (4.16), it is a standard procedure 


to assume that 


Pete) = KCC): Qe tat (4.17) 
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where K(t) is a random process independent of X when conditioned on 
(t,X,u(t)) and differentiable everywhere, 

Substituting equation (4.17) into equation (4.15) we get, 

) 

| 

5 (4.18) 
-1 

“Ry BY (t)E{K(t) |t,u(t) IX. | 


UR(t,X, u(t)) 


RE BY (t)ECK(t) [tu (e) 3X, 


ve (t,X, u(t)) 


Similarly substituting equation (4.17) into equation (4.16), we get 
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— K(t)BL (EYRE BY (t) EC K(t) | yu (t) IX (4.19) 

Since K(t) is conditionally independent of X, and is symmetric with the 
given boundary conditions, 
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where 

K(T) = 0 
Since the augmented matrix [A(t),B (t) BU (t) ] must be in one of a finite 
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E{K(t) | (t,u(t))e[f]} = K, (t) 


iva time t, “(t, wet elas then 


EAK(t)| Cae) yelil} = “AJK,(t) - K (tA, 


hi -1 
fb K,(t) [By Ry Ba By Ry "By 1K, (t) 


-R. (4.24) 


From (4.19) it follows that K(t) is bounded on [0,T] and consequently 
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A>0 
(4.22) 


The matrices A(t) ,B,(t) ,By(t) are Markovian. Therefore 
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The optimal controls are, 


A 


U* 


-] tt 

ay ug ye Ae 
-—l]_1 r 

“RY BY 4K, (tx | 
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Vx 


a 


Equations (4.25) and (4.26) are valid 
Pree. (t))e [7] 

The solution of the stochastic problem is given by (4.25) and 
(4.26). The gain matrices K,(t), i = 1,...,s, may be obtained by direct 
integration of equation (4.25). The realization of the optimal controls 
is shown in figure (4.1). These results are applied to a specific 
numerical example in the next section. 

We will conclude this section by making two observations. 
1) As a special case if ACE)=O;Vtele ei; then we get results for a 
class of Pursuit-Evasion problem considered by Ho, etal. (7!) 
2) If the matrices R, Rye and Ry are also subjected to sudden changes 
without violating their other properties mentioned before, the tech- 
nique presented in this section is applicable to include such situa- 


tions, provided that the vector u(t) carries complete information 


about the instantaneous values of A, By» BysRsRiy> and Ry. 


4.4 A Numerical Example 
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ordinary differential equation, 
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The function a(t) is a Markov jump process with two possible states, 
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nonzero probability that a(t+t) = 0, and t>0. Therefore the Q matrix 


peor act) “is 
Q= Guz 10 (4.28) 
and the initial probability distribution for a(t) is given by, 


Ree 0. Spa E (4529) 
| 1-p 


Let the cost scales be given by, 


ik \ 
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and b(t) = 1 and b(t) = v2, Vte[0,~) 


For this specific set of data the controller gains are solutions of, 


K, (t) 


- - che dotated - (4.31) 
K, (t) = -10K, (t) + (2-1)K, (t) = qk, (t) + qk, (t) - 1{ 


#(2-1)K, (t)*-1, 


For any finite value of T, the solution of (4.31) gives the best feed- 


back gains. However, we can make use of the stationary solutions of 


(4.31) because T>~. Thus we get, 
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u*(t,x,u(t)) = lim 1 K, (t)x ] 
TT? x 
' LE CECE) eta (4.32) 
ve(t,x,u(t))' = lim /2 K. (t)x 
T>© J 
eer) = 0 
we obtain lim K, (t) = +] (405) 
Too 
and 
ey 51 (t)) 8 
$f aCe). ="0 (4.34) 


v¥(t,x,u(t)) = -/2 x 
and this is the solution of the deterministic problem with a(t) = 0 
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co K(T3t 4x, ue, ve)? + vE(T3x(T5t, 4x jue, ve) ,u(r)* 
Pr { dt>p 
to — vi(t3x (rt) 4x ,uk,ve) u(r) * 
ShO) (4.38) 
In spite of this 
lim x(t5t »X,,u*,v*) = 0 (4.39) 


To 
for all initial conditions. This anomaly is due to the fact that 
controllability, lim x = 0, of the system when a(t) = 5 requires a 

T+ co 
lot of control "effort". Consequently if one expects the transition 
in a(t) to occur soon, only the "degree" of uncontrollability is 
maintained within bounds. For this specific example, this anomaly 
appears only for cases where a(t) > l. 

It was possible to obtain an analytic solution for the 
problem considered in section (4.3) because we considered a relatively 
simple objective function. It must be remembered, however, that for 
a more complex objective function, the search technique described in 


the previous chapter will have to be used. For instance suppose that 


the objective is of the form given by, 
= 4.4 
) El (2, 525923) op tox, (t)} ( 0) 


where z and Z, are as defined by equation (4.12). 


a2? 
As pointed out in the previous chapter we first solve the 


so-called linear combination problem (objective function 


= X ,u(t_)} treating the weighting factors 
E{ (c)2, + C424 Co2o) pan Cans u ( *) g 


Cy Co» and C3 as parameters. The negative sign is used to ensure 


that we have a min-max type of problem. 
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The expression of the Hamiltonian is given by, 


ACE)K + BL(t) U(e,X,u(t)) | 
Hie= PNCESX) 2 : | 
+ By(t) V(e,Xsu(t)) | 


i ' db ae ee ok as = 
fe 7 °y XeRX 9 CAV RLV 9 coU RU (4.41) 


The matrix Ricatti equations given in equation (4.20), now take 


the form 
K.(t) = -A'K,(t) = K,(t)A, 
462) 8, 4 
a ws —ly ci -liy a 
i a Bay BY; ron, Boy u By 1K, fe) 
33 q2 
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where er C39? and 3 are positive constants. 


The optimal controls are 
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problem and using the search technique described in the previous 
chapter. By substituting these numerical values in (4.42) and (4.43) 
then equation (4.42) and equation (4.43) give the solution of the 
stochastic problem. The gain matrices K te 1, =U) ..¢,sG0mayebe 


obtained by direct integration of equation (4.42). 


SHON It seh We 
enye * dimen} 
(EW USD) Big (SNP shin 

eats 30 oolsutod gaa | Hii 


aq 2am jay. say = id Lone a 
BB pits ne os a . a 


[1] 


[2] 


[3] 


[4] 


97 


REFERENCES 


Chattopadhyay, R., "On Differential Games", Int. J. Control, 
VOumpO,eNO. 3, (Dp. 26/-295, 1967. 

nO; YoChs abryson, AVE. Jr., and Baron, Sts" "Di tterential Games 
and Optimal Pursuit-Evasion Strategies", IEEE Trans. 
Automatic Control, Vol. AC=-10, pp. 385-3895 October®1965: 

Sarma, 1.G. and Ragade, R.K., ‘Some Considerations in Formulating 
Optimal Control Problems as Differential Games", Int. J. 
Conproime Vol. 4, No.3.) pp. '265-2795 19/662 

Sworder, D.D., "Feedback Control of a Class of Linear Systems 
with Jump Parameters", IEEE Trans. Automatic Control, 


Vol.) AC=14, pp. 9-14, Feb. 1969. 


ah aad a ' 
| eh a 
rie 
toainee Ul betel ‘ellie sii 
NOP BBG AROS: yey tala 
ESS), Lee IRS TAT. 9 4% AOter bela ee . BVA peed | 
‘Haeer Mae | Nilaaaaelal Hohanra~s duets Laatitg® ate e 
20S cromogor . OBE2GE ga VT. Tay dome aaah 
ge setigeght ni age PtebtaRey Sang" «yA sshngar hue aA | 
i Yat 9) asl yebindeaai sh atatduet Lorcdadod Kamit 7 
anor PTO Rae 92h 1 Wd,» KOV  Eoesaeo 
aaity Tidndd ln eeeth e845 teyeioy Sastbest” . 0-0, 


Whe 


vaigoy Sd ianea? BaAT ee ipece\, nid ow 


98 
CHAPTER (5) 


OPTIMIZATION OF A DETERMINISTIC SYSTEM WITH SEVERAL 


CONTROL FUNCTIONS AND SEVERAL COST FUNCTIONALS 


dee. introduction 

The problem considered in this chapter is concerned with 
the determination of optimal controls for a deterministic system with 
several independent controls and several cost scales (say p controls 
and N cost scales). Each control has several components. In other 
words each control is a vector. Each cost scale is a function of all 
controls. The controls will be assumed to be noncooperative in the 
sense that each control will seek to minimize or maximize its own 
objective function, each of which in turn is expressed as a function 
of the cost scales. In this chapter we will assume that each control 
is maximizing its objective function. Previous work relevant to the 
discussion here can be found in [2]. 

Examples where such problems would arise can be found in the 
field of Economics and Biology. There are some economic situations 


which can be viewed as non-cooperative-many-player processes [1,2]. 


5.2 Mathematical Formulation of the Optimization Problem 
Dynamics of the System 


Let the dynamics of the system to be controlled be represented 


by the following ordinary vector differential equation, 


X(t) = £(K(t),U, (t),---.U,(t)): a 


X(t.) = Xo 
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where (X(t) = dX(t)/dt).X is an n-dimensional vector representing the 
state of the system. f is an n-dimensional vector valued function 
continuous in all its arguments and continuously differentiable with 
respect to X.U;, i = 1,3s'.p, are the control variables, U, has a 
dimension rye The control vectors are required at each instant of 


time to satisfy certain boundedness constraints of the forn, 
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The control U, (t) is called an admissible control if, 
i) it is piecewise continuous on the control interval 
[t)>T], where T is the final time, and 
fi) it satisfies (5.2), vte[t»T]. 
The set of admissible controls is denoted by Us LS pel is. = si 
Cost Scales 
Let us define the following quantities, 
z= Six ee Re (5.3) 
which can be rewritten as, 
zy, = Sp£(X(t), Uz, (t),.--sU,(t)), ue 
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where Ze kf= ao. Ni are the cost "scales for the *systen- 5,8 are 


vectors with positive constant elements. 
Objective Function 
Let us define the objective functions as, 


os = $; (ayor ees 2y)> 1 = 1,.<«6,5pP e5) 


where ,'s are of class C. 
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The control problem can now be stated as follows, find 


the admissible control functions UUs such that 


* 4 * am 
a Ss BO aaa Ue Rie Lie : es rs : 
as O rie Uy 2 O,(T3X ete Ee Eel -+) 
(5.6) 
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As indicated earlier, if all the controls, except one, maintain their 
optimal strategies, then the one deviating from the optimum, may only 


lower the value of its objective function. 


Necessary Conditions for Optimality 
[2] 


Karvovskiy and Kuznetsov obtained necessary conditions 


for the system given in equation (5.1) for cost scales which can be 
considered as a special eee of equation (5.3). If we let 

Zy = Xy, Les erate origi (522) 
our objective functions will be the same as those treated in [2]. If 
we augment the cost scales equations given by (5.4) to the system 


equation (5.1) we can use the necessary conditions in [2] for the more 


general case considered here. 
From equations (5.1) and (5.4) we have, 


i f X(t.) | fx, | 


| 
: | = (5.8) 
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Where the augmented equation (5.8) is of dimension ntN. By applying 


the necessary conditions given in [2] we get, 
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Vte[t,,T] t= 1,+++5Ps 
where 
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P(t) = Eyal Che) 
N 
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P.(f) = ie ua, 7s (Sao) 
k=] . 
Equation (5.12) can be rewritten as, 
N 
P.(P) = ys Se, (5.13) 
[heel 


where C., are treated as parameters. Therefore it is clear that the 
problem with objective functions given by (5.5) is embedded in the 
linear combination problem. Equation (5.9) through (5.11) and equation 
(5.13) constitute the necessary conditions for the linear combination 
problem. 

As we have done in the previous chapters, the solution of the 
problem with objective functions given by (5.5) consists of two steps. 
The first step is to solve the problem with a linear combination of the 
cost scales as an objective function, treating the weighting factors 
as parameters, and the second step is to use the search technique des-— 


cribed in Chapter (1) to determine the optimum values of Cis i= pee 


J. Application 
Example (1) 


Let us consider a system described by the following equations, 
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and U5 = (uy 5) are constrained as follows, 
Wy a Uy 5 cae Wy > O and Uy oe 0 
(5.159 
0 S Uy, <1 
Cost Scales 
Let us consider a three cost-scale problem. Let 
24 = xy se 2X5 
Za = 2x1 ap Xy (5:16) 
Zz. = Xo 
Let, 
) SHO tZieZ4)snand 
if TRS LED CS 


ia VELS5 
Uy is maximizing 4 and U, is maximizing do: 


We will first solve the linear combination problem, by letting 


by = C4774 + C1929 and 


: (5.18) 
bo = Cy12> Cy, can be taken = 1 
For Uy 
Hy = Poy (axy + U41*1) + Poo (bx, SF Biot. az Un9%o 5-19) 
and 
3H. 
Poy _ son = (atu, ,)Po41 = Us 5Po9? Po 1) Py 0 
os (5.20) 
Mts ale antes T) 0) 
27k TET (btu) Poo Po 7) 


As a consequence of equation (5.9), assuming Xo is positive, we get 


x (5.21) 
Us 5 (t) re 


For UZ 


Hy = Py (a%) + U4 4%) + Py 9 (bx5 + Uy 9X] “F Uy 9X5 (5e22) 
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and 
F oH, 
= = = = + < = 
Pay af TAFT Pay 7 YppPi9> Pay hea rmtienetns 
: oH, Se 239 
=> - — = -. = 
Pi2 5 Ae ae eel: Py ee a 
As a consequence of equation (5.9), assuming xy is positive, we get 
uy, = O and Uo = ee SLE Piz < Pao 
uj, = 1 and Uo = Ons sate Piz? Pao (5.24) 
Uyy Fees and Uy if Py = Pyo Singular case 


The results obtained for this example are summarized in table (19 
Having solved the linear combination problem we can determine 


the optimum values of Cay and Cio for any specified objective function 


1 by using the search technique described in Chapter (1). Since this 
has been demonstrated in earlier chapters it will not be carried out 


here again. 


The preceeding simple example has illustrated to some extent 
the usefulness of the method developed in this chapter. It must be 
mentioned that more complex problems can be treated by this method. 
For instance a multilevel control problem is posed in example (2) and 


some preliminary results which have been obtained so far are reported. 


Example (2) A Multi Level Control Problem 
A multi level control system is considered in this example. 


See figure (5.1). The system has K levels, and level £(% = 1,...,K) 


has qo sectors. The lower levels sectors receive controls from the 


higher levels but not vice versa. However, there is interaction in 


both directions between sectors at the same level. Examples of such 
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Table (5.1) 
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Figarey(5.1)2A4 Multi-Level System 
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Systems may be found in the field of Economics. Karvovsky and 

Z A ; : : , 
pererecy | consider a single level system with several interacting 
sectors. Consequently, it will be possible to use their notation 


and terminology in the present discussion. 


Let the dynamics of the system considered be expressed as 


follows, 
bal 
X(g,m) ~ = eyes) Cees) 
s=l1 
do 
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s=l1 | 
: | 
dgenip i 
| 
is AU (ys) (25m) *(2,8) 
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qy dy 
ae OR B x 
i E Civaoieemtn Clas) €e a (K,s) (2,m)"(K,s) 
s=] s=l 
+ em) 
Suge a ym), 
Valet Ke m= 1y4- +54, (5.25) 


Some explanation of the significance of the symbols used seems to be 


in order here. 
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dimension ye 


AU xX is a column matrix which represents the contribu- 
(1,s)(2,m)° (1,5) 
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the state of sector (2,m). 


aCe) (Gem) is a rectangular matrix with a typical element 


“ss)y amy “(,5)/(my PEF 201,58), (2,m)z 


is a coefficient. 


u is a component of the required strate of sector 
oo Gs P oa BY 
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the rate of change of state x ke 


(2,m)=? * ney 


and Y = era 


Bei ey enn) a rectangular matrix with constant elements, is the 
> 3 
coupling matrix between state of sector (1l,s) and 


rate of change of the state of sector (2,m) 


Dep ) is a column matrix which represents the free terms. 
m 
> 


Figure (5.2) shows schematic representation for special case K = 2, 
q, = 1, and = 2. It must be noted that the arrow at the input 
represents the effect of the BX terms. 


It is assumed that the strategies of the sectors are constrained 


as follows, 
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As we have done several times before, we are going to con- 
sider first the problem with linear combination of the cost scales 
Efeating the weighting factors as parameters. Then by using the 
search technique described in Chapter (1) we can search for the optimum 
values of the weighting factors. 

For the probiem under discussion if we let the cost scales 
be the state variables of the sectors, the goals of the sectors 


(objective functions) shall be expressed as follows: 
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By od g 
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ee WE ey RS m=1,...,4, (5.27) 


Let us denote the part of the Hamiltonian ar) myer ee depends 
on the strategy of sector (2%,m) by Ee my Cony which is the only part 
subjected to maximization. Using equation (5.10) we get, 
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H a : 
(ijn) jm) Cay Chm), Gaon Cm)” 
| i 1 pax 
i i : 
l | | 
+ ° 
Wom Bh Ge) (5.29) 
B B 0 
where Ua "Ss (y = 1,...,8) are scalar functions. Here 8 denotes ayant 
We shall use this simpler notation from here on. 
From the constraints given in equation (5.26) we get, 
ue “(t,m)(2,m) * 
U U 
(2m) (k,p) ~~ (5m) 
Max *(2,m), v 
U 
(Cam) KP Cam) 
ie) ; 
| | | 
| | 
l | 
(5.30) 


*(2,m)g(ksp) (Lam) 


Let us assume that, the conditions, 
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are satisfied in the interval {0,T]). . This is a realistic assumption 
from a practical point of view. For instance if the system in equation 
(5.25) represents sectors of some economy, then the production of a 


Enh * 
sector will no doubt be positive. Consequently, for u (2,m)> (kp), Pa 
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On the basis of the linearity of SG. m)/at with respect to 
wed") 


X, the functions oF Cy m)/ at do not depend on X. The boundary condi- 
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Etons for P are, 
(2 ,m) 
12 a = G - jo = § 
(2m) (2,4) Cm) (5d) ae or 
| i 
| (5.32) 
| | 
P . = GC A . j = ‘ 
(24m) (Ky) ~ “(2,m) (K9) casita 


By integrating the system of differential equations of the adjoint 
system with the boundary conditions given in (5.32), the optimal 
strategies for each sector are determined from equation (5.31), 
Pale ss yh and m= Lyeeesdye The optimal strategies are used to 
determine an Cl 7) Vv atek and n= 1,+++5d,) by the forward in- 
tegration of equation (5.25). The search technique used in Chapter 
(1) may be used to determine the optimum values of the C's in equation 
(5.27) corresponding to the maximization of any objective functions 


d's. The problem considered in this chapter will be the subject for 


future research. 
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CHAPTER (6) 


CONCLUSIONS 


6.1 Summary 


In this thesis, we have been concerned with the optimization 
of systems with several cost functionals. Both single control and 
multi-control cases have been treated. While the discussion has been 
mostly restricted to deterministic systems, some results for systems 
with sudden changes in parameters are included. 

The main contribution of this thesis can be summarized as 
follows: 

It has been shown that the optimization of a system with 
respect to an objective function (which is expressed as a function of 
several given cost functionals) is embedded in the linear combination 
problem (i.e.) an optimization problem in which the performance 
criterion is a linear combination of the given cost functionals. 
According to the technique proposed in this thesis, the optimal con- 
trols for the linear combination problem are determined as a function 
of the weighting factors, and the optimum values of these weighting 
factors are obtained by a search technique for the given objective 
function. 

There are two main advantages of this technique, 

1. The solution of the linear combination problem can be used 
for any number of different objective functions (which are functions of 
the same given cost functionals). This technique can handle even case 


where the given objective function attains its extremum on the boundary 


of 1. 
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2. While previously reported techniques are mostly confined 
to cases where the single cost functional is expressed as a single 
integral, the technique proposed in the thesis is not restricted to 
such cases (see example in Chapter (3)). 

Another contribution of this thesis is the development of 
the modified accelerating step search technique used to solve the 
example in Chapter (3). Although it is not carried out in this thesis, 
this technique can be used to solve numerical examples in Chapter (4) 
also where the system is subjected to jump variation in its parameters. 

Perhaps a few words about the computational time required 
may be in order here. It is true that the computational time required 
to determine the entire manifold 1 is high. However, it must be 
remembered that this will be necessary only when the optimization has 
to be carried out with respect to many objective functions. In such 
a case, the time required to determine 1 is compensated by the fact 
that the linear combination problem need be solved only once. 

In the case where the optimization has to be carried out with 
respect to one objective function only, it is not necessary to deter- 
mine the entire manifold 7. Thus the computational time will be reduced. 
6.2 Suggestions for Future Work 

The preceding discussion is not intended to suggest that all 
unsolved problems have been treated in this thesis. The following 
are some of the unsolved questions which come to the author's mind. 
There must be several others. 

1. In Chapter (2), it is assumed that the form of the inte- 
's do not change during the course 


grands, assuming q,'8 are = 0, Le 


of the operation of the system. In other words, 
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However, situations may arise where this assumption may not be valid. 


In other words,' 
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Ht 
The Pelucion of such problems needs to be investigated. 

2. In the usal formulation of the Pursuit-Evasion problem 
it is assumed that ine dynamics of the evader is independent of the 
dynamics of the pursuer. However, it may be useful to consider 
Situations where the pursuer has some effect on the dynamics of the 
evader and vice versa. Such a problem can be handled by using the 
technique proposed in Chapter (3). A stochastic version of the pre- 
ceding problem is also worth considering. 

3. The work reported in Chapter (5) of this thesis relating 
to multi-control optimization problems should be pursued further by 
applying the technique to specific practical examples. 

4. For the multi level system treated in ReE CK CJ seit 
may be interesting to explore the feasibility of using Dynamic 
Programming to solve the optimization problem. The fact that the 
system not only has multi levels (which may be thought of as stages 


for purposes of Dynamic Programming) but also many sectors at each level 


might pose difficulties. 
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5. The question of using the Functional Analysis approach 
to the problems treated in this thesis also offers a rich area for 


future research. 
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APPENDIX 
Computer Programs (IBM 360/67 System) 
FORTRAN IV G LEVEL 1, MOD 4 MAIN DATE = T0241 21/50/42 
C THIS PFOGRAM TS CONSTDERED AS A TYPICAL EXAMPLE FOR ONE DIMENSIONAL 
C SFARCH.THF SYSTEM CONSIDERED IS REPRESENTED BY, 
c K1'=X2 
Fey G ie Lee oP Ae) eeeh a e 
C WITH INITTAL STATE( 1,1) AND FREE FINAL TIME.X1L(T)=X2(T)=0. 
c 
¢ COST SSCAWES JAREs 
G OT=TIME ELAPSED TO REACH (C,0),T 
C NF=FUFL CONSUMED TO REACH (0,0),F 
C 
@ UFO=ORJECTIVE FUNCTION, (OT*¥*R)*GE,R GREATER THAN ONE. 
C 
( A=R 
C B=C 2/01 
C NELI=INCREMENT OF (B) 
ee oT Lf DELZ=INCREMENT OF (A) . - 
G KK=NO OF RISFECTION STEPS=25 
C KA=NO OF DISCRETE VALUES QF B,yLESS THAN 100 
C KR=ENQM OF ATSCPETE VALUES OF A,yLESS THAN 100 
C THE MAIN PROGRAM MUST BE ASSOCIATED WITH A SUBROUTINE *PAR®*. 
C SUBROUTINE *"PAR* COMPUTES "OT*AND*OF*GIVEN C2/Cl. 
G MAIN PROGRAM AND SUBROUTINE ARE DOUBLE PRECISIONS 
occ l PFAL*™& BoeFeTyCyAgUFyUFMsNDX,B1,R2,83,0F1 »,OF2,0F3,0T1,OT2,0T3,UFOl, 
XUFON2,UFO3 AA GBR 
0092 NIMFNSTON F(169)2T(100),C(100),UF( 100) 
6003 REAN(S,1090)DEL1 »DEL2 »AyByKAyKB 
OC04 109 FORMAT (4F16.8,213) 
0005 NO 1 J=1,KBR 
0006 CALL PAR (8,F(JI),T(JI)) 
0007 C(JI=P 
9098 1 B=R4NELL 
0009 NO 5 T=1,KA 
0019 NO 2 J=1,KR 
OCcl] VE CS) =(T( J) F*A ) EF 
0012 2 WEITE(G,83IC( SI) FCS) 9 TIS) UFC II A 
CO1L3 8 FORMAT(5X,6F14.8) 
0014 JFM=IFE(1) 
9015 NO 32 J=1,51 
GLO Ie3 ITF (UF (SI)-UFM) 45,2353 
_O°17 4 UFM=UFCJ) _ 
0018 1C=J 
COL 2 CONTINUE 
C020 R2=(1C-1)*9EFLI 
0621 oy) aie ies aks 
0n22 BL=RZ2-NEL1/ (2 00 ¥*KK ) 
0023 BA=R240ELL/(2 OCO**KK ) 
CO24 CALL PAR(B1,0OF1,0T1) 
0025 CALL PAP (B2,0F2,0T2) 
0026 CALL PAR(R3,NF32,0T3) 
Nc?T UFOL= (NT1**A) XOF1 
0c 28 VER 2=(NT2**A ) XOF2 
0029 UFC A= (9T3*RA ) OFS —— 
0°30 AA=DAPS(UFEON2-UFOL) 
0031 PR=NABS (UFN3-UFND2) 


0032 TE ( CAA4BR)-529*(10.0* *(-16))) 3549355 36 
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(FORTRAN IV G LEVEL DATE = 79241 21/59/42 
| 
| 0033 36 IF(UFCL-UFO2) 30431432 
| 0034 30 IFC UFO2-UFO3) 33533435 
| 0035 31 LFCUFO2-UFO3) 34, 35,35 
, 0036 32 TF(UEO2-UFO3 ) 34934, 42 t. : <— pal re th ee 4 = 
| 0037 33 B2=Al 
0038 60. Toe29 
0039 34 B2=PR2 
0040 Conta 29 
0041 42 B2=B3 
| 0042 39 CONTINUE | ’ ; aes Pree 
0043 35 CALL PAR(B2,0F2,0T2) 
| 0044 UFO2=(0T2*#A) eQE? 
| O45 91=1.090000N09000000/82 
C046 Bl=B1*(OT2**A) 
C047 PR=AX (OT 2**(A-L.00) )*OF 2 
C CHANGE (UFO) /CHANGECOT )=-B1+B3 ie. : 73s 3 
0048 WRITE(6 910) KKyB1,R3 
0049 10 FORMAT(//,5%_1452F14.8) 
0050 WRITE (6_8)22,0F2,0T2,UFO2,A 
| 0051 WRITE (6,11) 
' 0052 11 FORMATCLHY) 
_ 6053 S A=AtNEL2 sees fen Ate ee b 
| 0054 END 
| 
= ’ an | 
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FORTRAN IV G LEVEL 1, MOD 4 PAR DATE = 70241 21/50/42 
0001 SUBROUTINE PAR(B,OF,OT) 

0002 REAL*B ByOF,OT,DyC 

0003 0=0.542.04R 
0004 C=DSQRT(3.0/(1.042.0%D) } 5 
0005 OF=1.04+2.04C 

0006 OT=1.0+(1.5+D) #C 

0007 RETURN ‘ 
0008 END 
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FORTRAN IV G LEVEL 1, MOD 4 MAIN DATE = 7024) 21/49/55 


THIS PROGRAM IS CONSIDERED AS A TYPICAL EXAMPLE FOR TWO DIMENSIONAL 
SEARCH.THE SEARCH TECHNIQUE USED IS THE ACCELERATING STEP TECHNIQUE 


THE SYSTEM CONSIDERED IS REPRESNTED BY, 


XL"=X2 
X2"= 
WITH INITIAL STATE(1,0)AND FREE TERMINAL TIME.X1L(TI=X2(T)=0. 


COSTS SCALES TARE, 
OT..=TIME ELAPSED TQ REACH (0,0) 


OF..-=FUEL CONSUMED TO REACH (0,9) 
QE..=ENERGY DISSIPATED TO REACH (0,0) 


OBJECTIVE FUNCTION=10.¥*(OT..-2.)**240F..+0F.6 
=COST OR XCOST 


0S81=C€2/C1 

OS2=C3/C1 

DDX=INCREMENT IN OF... DIRECTION 
DDY=INCREMENT IN OE. DIRECTION 
XOI=INITIAL STATE 


THE PROGRAM IS CONTROLLED TO THREE ITERATIONS,(CAN BE CHANGED). 
THE STEP SIZE IN EACH ITERATION IS= 10.*®*(-(IST(K))),K=1l_2,3. 

THE PROGRAM IS ASSOCIATED WITH A SUBROUTINE "PAR*. SUBROUTINE *PAR® 
COMPUTES 0S1,0S2,O0TeesAND XCOST ASSUMING OF. AND OF... ARE GIVEN. 
XGI=1.,IF XO1 IS CHANGED, *PAR*® MUST BE CHANGED ACCORDINGLT. 


0001 REAL*8 OS1,0S25XOI 5 0Fy OE» OT» GRAD 1, GRAD2 » COST »y XGRAD 1» XGRAD2,y XCOST 9D 
XDX~¢ ODVyNXy DY, OF Ny OENsOTN oY 9 Ay By OFC, CEC ,0F1 »OF2,0F3, 0El, OEF2,0E3,0T1 
X 90T2,0T340S11,0812,0S813,0821,0S22,0S8235XCOST1 » XCOST2, XCOST3 
0002 DIMENSION OF (3) ,OE(3) »OT(3),GRADI1(3),GRAD2 (3) ,COST(3),1S(3),T1ST(3) 
0003 READ(5,y500)NS1,0S2,XOI OF €1) yOE(1),0T(1) ,GRADIL(1),GRAD2(1),COST(1) 
XeIST(1),1ST(2]),0ST(3) 
0004 500 FORMAT(9F12.8,313) 
0005 XGRAD1=-GRAN1(1) 
0006 XGR AD2=—-GRAD2(1) 
0007 XCOST=COST(1) 
0008 WRITE(6,101) 
0009 101 FORMAT(10X%_*GRAD1",9X_ *GRAD2",10X,* FUEL °y9X_ "ENERGY" y 9Xy TIME 9 IX, 
X "LENGTH" 9 6X» "-EPS2/EPS1* 9 4X» *-EPS3/EPS1®* 9 /) 
0010 NO 15 JJ=1,3 
c 
c FQUAL STEP SEARCH 
C 
ooll WRITE(6,300) JSJ 
e] 00 FORMAT(5X,"I TERATION STEP NUMBER (*,IT1,*)* 
0013 JC=0 
0014 WRITE (6,8) GRAD1(1),GRAD2(1),OF(1),0E(1),OT(1L) eXCOST,OS1,0S2 
0015 8 FORMAT(5X_,8F14.8) 
0016 K=1 
0017 3 IF(XGRAD1)10,11912 
0 OQ DNX=-10,.0**(- 
0019 DDY=NDX*( XGRAD2/XGRAD1) 
0020 GOVLO els 
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_|FORTRAN IV G LEVEL 


| 


9022 


ly, MOD 4 | MAIN 


DATE 


70241 


i} 


21/49/55 | 


| 
| 


14 NDX=0.000000000 
0023 DDY=-10.0%#(=(IST(JJ))) | 
0024 GG To Tn3 | 
0025 16 NDX=0.c0000N000CN0 | 
| 0026 DDY=10.0%*(-(IST(JI))) 
| 0027 CGlTOsIs 
0028 12 DDX=10.0**(=(IST(Jd))) 
0029 DNY=DNDX* (XGRAD2/XGRAD1) 
0030 13 NO 1 I=1,1000 
0031 DX=DDX* 1 
; 0032 DY=DNY* I 
0033 OFN=0F(K)+NX 
| 0034 OEN=NE(K) 4ny 
0035 Y=XCOST 
0036 CALL PAR (OFN,QEN,9S1,9S2,0TNyXCOST) 
0037 OS1=-9S1 
9038 0S2=-0S2 
0039 IF(XCOST-Y) 19292 
0049 1 CONTINUE | 
ie 
G BISFECTION SEARCH 
| C 
“0041 ? OFC=0FN-DDX : 
| 0042 NFC=OEN-NDY 
0043 DO 39 KK=1,12 
0044 OF2=O0FC 
0045 OF1=OF2-DNXK/(2.00000000000**KK) 
0046 OF3 =OF24NDX/(2.00000000000000**KK ) aa 
0047 OQE2=0EC 
0048 OF1L=NE2-NNY/ (2.000000000000000**KK ) 
| 0049 NE3=0F24+NNY/ (2. 0000000000000000*#KK ) 
0050 CALL PAR(OF1,0£1,0S11,0521,0T1,XCOST1) 
0051 CALL PAR(OF2,0E2¢0S12,0S22,0T2, XCOST2) 
0052 CALL PAR (NF 3,0£3,0S13,0S2 3,013, XCOST3) 
0053 A=DARSUXCOST 2-XCOSTL) 
0054 B=DABS(XCIST3-XCNST2) 
0055 TFE( (A#B)-5.0%(10.0%*(-16))) 35,354 36 | 
0056 36 LFE(XCOST1-XCOST2) 30931532 
0057 30 LFCXCOST 2-XCOST3)33940.40 
C058 31 IF(XCOST2-XCOST3) 34740940 
0059 32 LF(XCOST 2-XCOST3) 345 34942 
| 0060 33 OFC=NF1 
0061 OEC=0F1 
0062 Goet09 
0063 34 OFC=0F2 
0064 OEC=0F2 within 
0065 GO TN 39 | 
0066 42 OFC=OF3 
0067 OFC =0E3 
0068 GONG 29 
0069 4D WRITE(6,41) 
0C70 41 FORMAT(2X,*SOMETHING WRONG IN THE BISECTION SEARCH*) 
0071 GO TO 35 
0072 39 CONTINUE 
' 0073 35 OFN=OFC 
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FORTRAN IV G LEVEL 1, MOD 4 MAIN DATE = 70241 21/49/55 
0074 OEN=OEC 
0075 CALL PAR(OFN,OEN,0S1,0S2,0TN, XCOST) 
0076 0S$1=-0S1 
0077 Q$2=-0S2 
0078 IF ( JC-1) 22,23,23 
0079 22 IS(K)=I 
' 0080 IF(K-2)44,5,5 
| 0081 4 OF(2)=0FN 
0082 O£(2)=0EN 
0083 OT(2)=0TN 
0084 GRAN1 (2) =20.0000000000*( OTN-2.00000000000) *0S514#1.00000000000000 
0085 GRAD2(2)=20. CO000000000*( 0 TN-2. 0000000000) *0$ 24+1.0000000000000 
0086 XGRAD1=-GRAD1(2) 
0087 XGRAD2=-GRAD2(2) 
0088 WRITE (6_8)GRAD1(2)yGRAD2( 2) ,OF (2) yOE(2) yOT(2) » XCOST »OS1,0S82 
0089 K=2 Ps 
0090 G0 To 3 
0091 5 WRITE(6,102) OF Ny OENy OTNy XCOST 081,082 
0092 k 102 FORMAT(33X,6F14-8) 
0093 XGRADI=OFN-OF(1) 
0094 XGRAD2=O0EN-0E(1) 
0095 XCOST=COST(1) 
0096 JC=1 
0097 K=JC 
0098 GO TO 3 pas _ aes 
0099 23 GRAD1(1)=20.000000000*(0TN-2-. 000000000) *0S1+#1.0000000000000000 
9100 GRAD2 (1)=20.900000000000*( OTN-2. 00000000000) #0$24+1. 000000000000 
0101 $(3)=1 
| 0102 WRITE(6,8)GRAD1L(1),GRAD2(1) sQFN,OENsOTNy XCOST,0S140S2 
0103 WRITE (6,104) I1S(1)91S(2)5IS03) 
0104 104 FORMAT(5X,1555X_ 1595Xy15) 
0105 A=DABS(GRAD1(1))+#DABS(GRAD2(1)) 
0106 IF(A-020090000000005) 17917524 
| 0107 24 OF(1)=0FN 
0108 O£&(1)=O0EN 
0109 OT(1)=OTN 
| 0110 COST(1)=XCOST matte 
0111 XGRAD1=-GRAD1(1) 
0112 XGRAN2=-GRAD2(1) 
0113 15 WRITE(6,100) ee 
0114 100 FORMAT(///) 
0115 Go. To. 7 
0116 (18 A=-XGRAD1 
C117 B=-XGRAD2 
0118 WRITE (679) AgB yOFNy OEN¢OTNyXCOST yOS1 4 O0S29Ke JSC 
0119 9 FORMAT(5X 48F14.8 
0120 60 TOL20C 
| D121 17 A=-XGRAD1 
| 0122 B=-XGRAD2 
0123 WRITE(6 9400) A 9B 
| 0124 400 FORMAT(1H1,2F20.14) 
9125 00 A=A ee ee ee ee ee 
END 
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FORTRAN IV G LEVEL 1, MOD 4 PAR DATE = 70241 21/49/55 
/ 0001 SUBROUTINE PAR(OFN, OF Ny OS1,0S2,0TN, XCQST) 
/ ©002 RFAL*3 OFN,NEN,OTN,OS1,0S2, XCOST 
' 90003 OS 2=2.0*(1.9-CEN/OFN) 
0004 CS1=2 -C0/( OFN*GEN)40S2*0S52/6.0-DS2-C.500 gin ae 
0005 OTN= (12049514052 )*0FN 
| 0006 XCOST=19- 00ND 0GOO000GO*( OTN-2. 00000000) *(CTN-2 .00900000) +OFN+0EN 
0007 RETURN 
OCO8 END 
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FORTRAN IV G LEVEL 1, MOD 4 MAIN DATE = 70241 22/08/09 


THIS PROGRAM IS CONSIDERED AS A TYPICAL EXAMPLE FOR A SADDLE-POINT 
SEARCH.THE TECHNIQUE USED IS A MODIFICATION OF THE ACCELERATING 
STEP SEARCH TECHNI QUE,REFER TO THESIS. 
YSTEM CO fe R 
X*sA*X+B UCC #Y 


THE COST SCALES COSIDERED ARE, 
EX1=O-.5*(FINAL STATE) **2 
EX2=O0.5*(ENERGY DISSIPATED BY CONTROL U) 
EX3=0-5*(ENERGY DISSIPATED BY CONTRCL V) 


OBJECTIVE=RMIN MAX(EX1+(EX3)/(EX240.5)) »V MIN AND U MAX 
=MIN MAX( OF), (SUBROUTINE*PAR® ) 


aa Aan 


EPS1=C2/C1 
| ¢ EPS2=C3/C1L 
KK=+10R-1 +1 FOR STEPS IN GRADIENT DIRECTION 
-1 FOR STEPS OPPOSITE TO GRADIENT DIRECTION 
GRAD1=D( OF )/D(EX2) 
GRAD2=D(0F)/D(EX3) 
OF 1=(GRADL*GRAD1+GRAD2*GRAD2)», ( SUBROUTINE *PAR®) 


Cc THE PROGRAM IS CONTROLLED TO THREE ITERATIONS(CAN BE CHANGEC). 


AAAaAo 


——— 


Cc THE STEP SIZE IN EACH ITERATION=(10.**(-IC(K))),K=1,2,3 
Cc 
C THE MAIN PROGRAM IS ASSOCIATED WITH THREE SUBROUTINES, *STEP* »*PAR® 
C sAND* BISEC*.SUBROUTINE*STEP® IS TO DETERMINE THE STEPS IN THE EX2 
Cc AND EX3 DIRECTICONS.SUBROUTINE *PAR* IS TO DETERMINE EPS1,EPS2,0F, 
A MIN X2 AND EX3 ARE VEN.SUBROUTINE "BISEC® A 
Cc BISECTION SEARCH. 
0001 REAL*8 XS1,XS29XS3eEPSI gEPS2 OF »X19X29X3,GRADL »GRAD2 » 0X2, DX3,D2,03 
‘ee  XgX IN yX2NyX3NeVeOFT SY Y,OFF 
0002 REAL*8 A yBeCoTeX09XSoCSoBS oQly O29 049059067077 08y Q95Q10,0F1,Al,A2 
| 0003 REAL*8 DEXP,DABS,DSQRT 
000% DIMENSION X1(3) 9X2(3).X%3(3),1C(3), ICCC3) 
0005 REAL (5,500) AeBeCeoX0y T,EPSL,EPS2,IC(1),1C(2) ,1C(3) | 
0006 500 FORMAT(7F14.8,313) 
0007 XS=X0O*#XO 
0008 CS=C*C 
| 0009 BS=B*B 
_ 0010 Q1=DEXP(-A*T) 
ooll Q2=DEXP( -2.*A*T) 
0012 €4=1.00000 
0013 Q5=EPS2*CS-EPS1*BS 
| 0014 Q6=Q5*Q4 
| 0015 Q7=Q6+( 1 -00000-Q6) *Q2 
Ui ete ee eee en 000 0 / OT a ae ee 
0017 Q9=Q8*C8 | 
0018 Q10=XS*Q9*C4*(1.00000000-Q2) 
0019 XS 2=0. 5000000*BS¥#EPS1#*EPSL#Q10 | 
0020 XS3=0. 5000000*C S*¥EPS2*EPS24#Q10 | 
0021 DO 15 IK=1,53 | 
joo22 CALL PAR (XS2aXS3.EPSLSEPS2eXSLeOF OF) 
0023 X2(1)=XS2 
0024 X3(1)=XS3 
G02S s,s X1(1)=XS1 
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FORTRAN IV G LEVEL ly MOD 4 MAIN DATE = 70241 22/08/09 | 
| 0026 A1l=X3(1) 
0027 A2=X2(1)+0.50000000000000 
0028 GRAD1=1.CO00000000/E PS1-Al1/(A2*A2) 
0029 GRAD2=-1.00000000000 /EPS2+1.0000C00000000/A2 aoa) 
0030 WRITE(6,8)GRAD1L»,GRAD2,X1(1),X2(1)9X3(1),OF 9 EPS1y,EPS2 
0031 8 FORMAT(5X,8F14.8) 
0032 KK=1 | 
0033 CALL STEP(GRAD1 »GRAD2,KKy IC( IK ),DX2, 0X3) | 
0034 OFT=OF 1 . 
0035 YY=OFT j 
| 0036 XS2=XS24+DX2 
0037 XS3=XS34+DX3 
0038 CALL PAR (XS29XS3eEPS1,EPS2,XS1y,0FF,0FT) 
0039 IF (OFT-YY)11,12,12 
0040 12 KK=-1 | 
_0041 DX2=-Dx2 | 
0042 DX3=-Dx3 | 
0043 11 DO 1 1=1,1000 | 
0044 C2=Dx2*! 
0045 C3=0X3*I 
0046 X2N=X2(1)4+02 
0047 X3N=X3{1)+D3 
| 0048 Y=OF1 
| 0049 CALL PAR (X2NyX3NyEPS1Ly,EPS29X1Ny0F ,OF1) 
0050 IF(OFI-Y)1,2,2 
0051 1 CONTINUE 
| 0052 2 CALL BISEC (X2N¢X3NyDX2,DX3) 
0053 100 ICCtl)=I 
' 0054 X2(2)=X2N 
0055 X3(2)=X3N 
0056 CALL PAR (X2(2)4X3(2),EPSLyEPS2,X1(2) »OF,OF1) 
0057 A1=X3(2) 
0058 A2=X2(2)+0.5000000000000 
0059 GRAD1=1.COCOOQO0000/EPS1—AlL/(A2*A2) 
0060 GRAD2=-1.~.0C00000000/ EPS2+#1.~0000C00000/A2 
0061 WRITE(6,8)GRADI yGRAD29X1(2)9X2(2)4X3(2),0F yEPSLyEPS2 
0062 KK=1 ' 
0063 CALL STEP (GRAD1L»GRAD2,KKy IC( IK) »DX290X3) | 
0064 OF T=0F 1 
0065 YY=OFT ace Ee es a 
0066 A1L=X3(2) #DX3 | 
0067 A2=X2(2)+DX2 
0068 CALL PAR (A2,AlLy,EPSLyEPS2+XI1NeOFF,OFT) 
0069 IF (OFT-YY1110,120,120 
0070 120 KK=-1 
oo71 DX 2=-Dx2 — 
0072 DX3=-DXx3 
0c 73 110 DO 3 I=1,1C00 
0074 D2=DX2*I 
C075 D3=DXx3*! | 
0076 X2N=X2(2)+D2 
_0077 X3N=X3(2)+D3 oe ed 
0078 Y=0F1 | 
oc79 CALL PAR (X2NyX3NeEPSL»>EPS2,X1NeOFsO0F1) 
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/FORTRAN IV G LEVEL 


| ly MOD 4 MAIN DATE = 70241 22/08/09. 
_ ocal 3 CONTINUE 
| 0082 5 CALL BISEC (X2NyX3N90X2,DX3) 
0083 200 ICC{2)=1 
0084 X2(3)=X2N | 
0085 X3(3)=X3N 
| 0086 CALL PAR (X2(3),)X3(3),EPSL,EPS2,X1(3) »OF yOF 1) 
| 0087 GRADL=X2(3)-X2(1) | 
0088 GRAD2=X3(3)-X3(1) 
0089 WRITE (6993X1(3) 9X2(3)9X3(3) OF EPS 1y EPS2 | 
_0090 9 FORMAT(33X,6F14.8) 
0091 KK=1 
0092 CALL STEP(CRAD1 yGRAD2yKKy IC( IK)» DX2y9NX3) 
0093 CALL PAR(X2(1)9X301) 9 EPS1yEPS29X1¢1) »OF,0F1L) 
0094 DO 4 I=1,1000 
0095 D2=DX24"I 
0096 D3=DX3*1 4 
0097 X2N=X2(1)+#D2 
0098 X3N=X3(1)+D3 | 
0099 Y=OF 1 . 
0100 CALL PAR (X2NyX3Ny EPS1lyEPS2,X1NyOF,OFL) 
0101 {EAOFIAY 16.9646 
0102 4 CONTINUE. 4 
0103 6 CALL BISEC (X2NyX3Ny DX2y 0X3) | 
0104 ICC(3)=1 
0105 CALL PAR (X2N¢X3NyEPS1yEPS2,XLNyOF,OFL) 
— O1C6 A1L=X3N 
‘orc? A2=X2N4#0.500000000000 


GRAD1=1.0000Q00000000/EPSI—AL/(A2*A2) 


15 


GRAC2=-1 .COCIODDDNNDDDO/EPS24+1.0000C000000/A2 
WRITE (68) GRADL»GRAD29X1INs X2N_X3Ny0F gEPSLyEPS2 
WRITE(6,7)1CC(L),1ICC(2),1CC(3) 
FORMAT (2X53110) 

XS2=X2N 

XS 3=X3N 
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“FORTRAN IV G LEVEL 


E27. 


1, MCD 4 ; STEP DATE = 70281 


22/08/09 


_ 0001 SUBROUTINE STEP (AyByKKyIyDAyDB) 
0002 REAL*8 AyByDAyDByC yDyE 
0003 REAL*8 DEXFsDABS,DSQRT 
0004 1F(A)10,11y10 be rereeetticeesl 
| 0005 11 IF(CABS(A)-0.0000000000000005) 20,20,21 
— 0006 21 IF(B)10,18,10 | 
0007 10 C=CSQRT(1.+(B/A)*(B/A)) | 
00068 E=DSQRT(1.4+(A/B)*(A/B)) | 
- 0009 D=10.0**(-1) 
__0010 DA=KK*D*(A/DABS(A)1/C 
| 0011 DB=KK*D* (B/DABS(B))/E 
| 0012 GO TO 19 
0013 20 DA=0.C00C00000000000 
0014 CB=KK*D 
0015 GO TO 19 | 
__ 0016 18 WRITE(6,8) oi 
0017 8 FORMAT(2X,*THE GRADIENT IS VANISHED") 
0018 19 RETURN 
0019 END | 
| 
| Bar)... an 
| 


128 


FORTRAN IV G LEVEL ly MOD 4 PAR DATE = 70241 22/08/09 | 
00c1 SUBROUTINE PAR(EX2,EX3,EPS1,EPS2,EX1,0F,0F1) | 
0002 REAL*8 AyBeCo To X09 XS CS 9BS 9Q19Q2903904,EPS1 gy EPS2 4Q5,05,Q79Q8y Q9, EX | 

X1_9Q10,EX2 yEX3yOF eR1gR2yR3yR4yR5 y¥GRAD1LyGRAD2 OF ly Aly A2 | 
0003 REAL*8 DEXP,yDABS,DSQRT | 

/ 0004 A=0.5000C000 | 
0005 B=0.0500C0000000000 
0006 C=0.0510CC000000000 sae 
0007 T=2.00000000 | 

- 0008 X0=1.500090000 
0009 XS=XO*XO ee) 
0010 CS=C*C | 
0011 BS=B*B 
0012 Q1L=DEXP{-A*T) 
0013 Q2=DEXP{-2.00*A*T) | 
0014 Q4=1.00000/12.000000%A) 

0015 Rl=( B/C) *DSQRT(EX3/EX2) ee: | 
0016 R2=BS*XS*0.500000000*(1.0000000-Q2) 

| 0017 R3=(R1*CS-BS)*(1-009000000000-Q2) 

0018 R4=DSQRT(EX2/R2) 

0019 R5=1.0000000/R4 

0020 EPS1=Q2*(1.00000000/ (R5—-R3)) 
EPS2=EPS1*R1 

0022 Q5=EPS2*CS-EPS1*BS 

| 0023 Q6=Q5*04 

| 0024 C7=Q64+(1.00000-Q6) *Q2 
0025 Q8=1.000000/Q7 | 
0026 Q9=Q8*Q8 

| 0027 EX1=0.50000*XS*Q2*Q9 

| 0028 A1=EX3 

- 0029 A2=EX2+0.5000C000000000 
0030 GRAD1=1~.CO000000000/EPS1-Al/ (A2*A2) | 
0031 GRAD2=-1.~.C000000000/EPS2+1.-.00000000000/A2 
0032 OF 1=GRAD1*GRADL+*GRAD2*GRAD2 | 

20033. |) ee DESEXT HAL AZ 
0034 RETURN 
0035 END : 
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FORTRAN IV G LEVEL ll, MOD 4 BISEC DATE = 70241 22/08/09 
oocl SUBRCUTINE BISEC (Ay 8,CA,0B) 
0002 REAL*3 AsBrDAs0BrOXs DV 5 XCy VC eX 1 9X25X34V1,V¥22Y350S11,08 214,05 22,0813 
Xy0S23,21,229239V19V25V3,0S12,VGl1 ,V02,V03 
— 0003 i __REAL*3 DEXP,DABS,DSQRT__ 
OO0C4 DX=DA 7 
0005 DY=CB 
00C6 XC=A 
ooc7 YC=B 
0008 Ce 39 K=1,40 
0009 1X22 XG ae CS. eee 
0010 X1=X2-NX/(2.000000000#*K) a ce’ 
OOll X3=X2+DX/(2.CCO000CO000**K ) 
0612 Y2=YC 
0013 YL=Y2-DY/(2.000OWNONe*L) 
0914 Y¥3=Y2+DY/(2.000000000%*K ) 
» DONE Ts ___€ALL PAR (X1,Y1L,0S11,.0S21,21,VO1,V1) ' 
0016 CALL PAR (X2,Y2,0S12,0S22922,V02,V2) 
OOl7 CALL PAR (X3+Y3,0S1350S23923,V%3,V3} 
0018 A=CABS(V2-V1) 
0019 P=DABS(V3-V2) 
0020 IF ((A+B)-5 .0*(10.0¥**(-16)) )40,40,36 
HO2M oni GEV V2) 3 Os Silico 2am bia a Oren ae 
0022 30 IF(V2-V3) 33, 33,40 
0023 31 IF(V2-V3)34,40,42 
0024 32 LFE(V2-V3) 34934,42 
C025 AB XG=K1 
0026 yc=Yl 
002i ei) eh 3) - 
0028 340 XG x2 
0029 YC=Y2 
0030 Goumona9 
0031 42 XC=X3 
0032 YC=Y3 
0033 a 39 CONTINUE a) eye tte roe om o et. 4 
0034 40 A=XC 
0035 R=YC 
0036 w2ITTE(6,8)K 
0037 8 FORMAT(2X,110) 
0038 RETURN 


200395 SEND ES = Seek 1 ee ee a = = Es 
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